Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_no_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_no_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 400,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 400,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_no_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.321152
Epoch 1
Validation binary_cross_entropy = 0.348233
Epoch 2
Validation binary_cross_entropy = 1.268520
Epoch 3
Validation binary_cross_entropy = 0.611619
Epoch 4
Loss = 7.0310e-01, PNorm = 31.6240, GNorm = 10.0708, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.307598
Epoch 5
Validation binary_cross_entropy = 0.328184
Epoch 6
Validation binary_cross_entropy = 0.678415
Epoch 7
Validation binary_cross_entropy = 0.498388
Epoch 8
Validation binary_cross_entropy = 0.329973
Epoch 9
Loss = 5.7544e-01, PNorm = 31.6788, GNorm = 10.0606, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.349349
Epoch 10
Validation binary_cross_entropy = 0.659491
Epoch 11
Validation binary_cross_entropy = 0.469829
Epoch 12
Validation binary_cross_entropy = 0.317995
Epoch 13
Validation binary_cross_entropy = 0.331606
Epoch 14
Loss = 3.0823e-01, PNorm = 31.7492, GNorm = 4.5470, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.427332
Epoch 15
Validation binary_cross_entropy = 0.525570
Epoch 16
Validation binary_cross_entropy = 0.439640
Epoch 17
Validation binary_cross_entropy = 0.397463
Epoch 18
Validation binary_cross_entropy = 0.413822
Epoch 19
Loss = 2.1781e-01, PNorm = 31.8093, GNorm = 3.1831, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.440949
Epoch 20
Validation binary_cross_entropy = 0.403231
Epoch 21
Validation binary_cross_entropy = 0.374261
Epoch 22
Validation binary_cross_entropy = 0.382577
Epoch 23
Validation binary_cross_entropy = 0.395020
Epoch 24
Loss = 1.1798e-01, PNorm = 31.8579, GNorm = 1.7431, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.425146
Epoch 25
Validation binary_cross_entropy = 0.437176
Epoch 26
Validation binary_cross_entropy = 0.387413
Epoch 27
Validation binary_cross_entropy = 0.373565
Epoch 28
Validation binary_cross_entropy = 0.359067
Epoch 29
Loss = 1.4310e-01, PNorm = 31.9015, GNorm = 2.5925, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.357145
Epoch 30
Validation binary_cross_entropy = 0.387465
Epoch 31
Validation binary_cross_entropy = 0.369225
Epoch 32
Validation binary_cross_entropy = 0.395707
Epoch 33
Validation binary_cross_entropy = 0.399877
Epoch 34
Loss = 9.1926e-02, PNorm = 31.9450, GNorm = 2.5250, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.411769
Epoch 35
Validation binary_cross_entropy = 0.419291
Epoch 36
Validation binary_cross_entropy = 0.414081
Epoch 37
Validation binary_cross_entropy = 0.422541
Epoch 38
Validation binary_cross_entropy = 0.437284
Epoch 39
Loss = 8.6349e-02, PNorm = 31.9845, GNorm = 2.0280, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.453125
Epoch 40
Validation binary_cross_entropy = 0.485541
Epoch 41
Validation binary_cross_entropy = 0.500453
Epoch 42
Validation binary_cross_entropy = 0.508984
Epoch 43
Validation binary_cross_entropy = 0.493442
Epoch 44
Loss = 5.0338e-02, PNorm = 32.0178, GNorm = 0.7447, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.466406
Epoch 45
Validation binary_cross_entropy = 0.497543
Epoch 46
Validation binary_cross_entropy = 0.468161
Epoch 47
Validation binary_cross_entropy = 0.489796
Epoch 48
Validation binary_cross_entropy = 0.478114
Epoch 49
Loss = 7.1297e-02, PNorm = 32.0497, GNorm = 1.6028, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.468289
Epoch 50
Validation binary_cross_entropy = 0.469644
Epoch 51
Validation binary_cross_entropy = 0.479961
Epoch 52
Validation binary_cross_entropy = 0.498451
Epoch 53
Validation binary_cross_entropy = 0.471929
Epoch 54
Loss = 1.1704e-01, PNorm = 32.0847, GNorm = 3.2406, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.449429
Epoch 55
Validation binary_cross_entropy = 0.490583
Epoch 56
Validation binary_cross_entropy = 0.502599
Epoch 57
Validation binary_cross_entropy = 0.487063
Epoch 58
Validation binary_cross_entropy = 0.513281
Epoch 59
Loss = 8.5593e-02, PNorm = 32.1204, GNorm = 1.9183, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.488775
Epoch 60
Validation binary_cross_entropy = 0.465864
Epoch 61
Validation binary_cross_entropy = 0.485631
Epoch 62
Validation binary_cross_entropy = 0.496104
Epoch 63
Validation binary_cross_entropy = 0.480679
Epoch 64
Loss = 7.4897e-02, PNorm = 32.1526, GNorm = 2.9492, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.519062
Epoch 65
Validation binary_cross_entropy = 0.517976
Epoch 66
Validation binary_cross_entropy = 0.498294
Epoch 67
Validation binary_cross_entropy = 0.497816
Epoch 68
Validation binary_cross_entropy = 0.507063
Epoch 69
Loss = 6.8535e-02, PNorm = 32.1831, GNorm = 0.7089, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.513932
Epoch 70
Validation binary_cross_entropy = 0.543978
Epoch 71
Validation binary_cross_entropy = 0.531160
Epoch 72
Validation binary_cross_entropy = 0.476159
Epoch 73
Validation binary_cross_entropy = 0.467599
Epoch 74
Loss = 5.0748e-02, PNorm = 32.2130, GNorm = 0.9452, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.472690
Epoch 75
Validation binary_cross_entropy = 0.495215
Epoch 76
Validation binary_cross_entropy = 0.526487
Epoch 77
Validation binary_cross_entropy = 0.575253
Epoch 78
Validation binary_cross_entropy = 0.560109
Epoch 79
Loss = 6.5188e-02, PNorm = 32.2417, GNorm = 1.9759, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.531139
Epoch 80
Validation binary_cross_entropy = 0.495136
Epoch 81
Validation binary_cross_entropy = 0.482712
Epoch 82
Validation binary_cross_entropy = 0.492759
Epoch 83
Validation binary_cross_entropy = 0.509707
Epoch 84
Loss = 5.6924e-02, PNorm = 32.2688, GNorm = 1.1976, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.586826
Epoch 85
Validation binary_cross_entropy = 0.635665
Epoch 86
Validation binary_cross_entropy = 0.606027
Epoch 87
Validation binary_cross_entropy = 0.613924
Epoch 88
Validation binary_cross_entropy = 0.656985
Epoch 89
Loss = 3.7263e-02, PNorm = 32.2961, GNorm = 0.8150, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.664251
Epoch 90
Validation binary_cross_entropy = 0.633072
Epoch 91
Validation binary_cross_entropy = 0.606828
Epoch 92
Validation binary_cross_entropy = 0.583585
Epoch 93
Validation binary_cross_entropy = 0.583693
Epoch 94
Loss = 4.9596e-02, PNorm = 32.3211, GNorm = 1.6424, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.612171
Epoch 95
Validation binary_cross_entropy = 0.638995
Epoch 96
Validation binary_cross_entropy = 0.664737
Epoch 97
Validation binary_cross_entropy = 0.693993
Epoch 98
Validation binary_cross_entropy = 0.613355
Epoch 99
Loss = 6.6265e-02, PNorm = 32.3474, GNorm = 1.5363, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.559634
Epoch 100
Validation binary_cross_entropy = 0.551373
Epoch 101
Validation binary_cross_entropy = 0.565144
Epoch 102
Validation binary_cross_entropy = 0.558057
Epoch 103
Validation binary_cross_entropy = 0.597405
Epoch 104
Loss = 3.0406e-02, PNorm = 32.3771, GNorm = 1.3517, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.650358
Epoch 105
Validation binary_cross_entropy = 0.665500
Epoch 106
Validation binary_cross_entropy = 0.651644
Epoch 107
Validation binary_cross_entropy = 0.640513
Epoch 108
Validation binary_cross_entropy = 0.632621
Epoch 109
Loss = 3.1383e-02, PNorm = 32.4017, GNorm = 0.7773, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.624915
Epoch 110
Validation binary_cross_entropy = 0.624000
Epoch 111
Validation binary_cross_entropy = 0.630865
Epoch 112
Validation binary_cross_entropy = 0.642908
Epoch 113
Validation binary_cross_entropy = 0.663620
Epoch 114
Loss = 2.4907e-02, PNorm = 32.4215, GNorm = 1.2333, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.672250
Epoch 115
Validation binary_cross_entropy = 0.649028
Epoch 116
Validation binary_cross_entropy = 0.666038
Epoch 117
Validation binary_cross_entropy = 0.675641
Epoch 118
Validation binary_cross_entropy = 0.632641
Epoch 119
Loss = 1.3673e-02, PNorm = 32.4421, GNorm = 0.2570, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.607931
Epoch 120
Validation binary_cross_entropy = 0.627112
Epoch 121
Validation binary_cross_entropy = 0.651999
Epoch 122
Validation binary_cross_entropy = 0.633831
Epoch 123
Validation binary_cross_entropy = 0.661266
Epoch 124
Loss = 3.6956e-02, PNorm = 32.4647, GNorm = 0.4383, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.755785
Epoch 125
Validation binary_cross_entropy = 0.770086
Epoch 126
Validation binary_cross_entropy = 0.731193
Epoch 127
Validation binary_cross_entropy = 0.716058
Epoch 128
Validation binary_cross_entropy = 0.694642
Epoch 129
Loss = 4.7880e-02, PNorm = 32.4872, GNorm = 0.3885, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.658611
Epoch 130
Validation binary_cross_entropy = 0.649245
Epoch 131
Validation binary_cross_entropy = 0.673016
Epoch 132
Validation binary_cross_entropy = 0.725180
Epoch 133
Validation binary_cross_entropy = 0.694255
Epoch 134
Loss = 5.2156e-02, PNorm = 32.5122, GNorm = 1.5839, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.621891
Epoch 135
Validation binary_cross_entropy = 0.637034
Epoch 136
Validation binary_cross_entropy = 0.669806
Epoch 137
Validation binary_cross_entropy = 0.638303
Epoch 138
Validation binary_cross_entropy = 0.623120
Epoch 139
Loss = 3.1122e-02, PNorm = 32.5371, GNorm = 2.4363, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.644358
Epoch 140
Validation binary_cross_entropy = 0.674206
Epoch 141
Validation binary_cross_entropy = 0.712657
Epoch 142
Validation binary_cross_entropy = 0.734486
Epoch 143
Validation binary_cross_entropy = 0.752105
Epoch 144
Loss = 6.0542e-02, PNorm = 32.5598, GNorm = 1.7174, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.791965
Epoch 145
Validation binary_cross_entropy = 0.755180
Epoch 146
Validation binary_cross_entropy = 0.705230
Epoch 147
Validation binary_cross_entropy = 0.741106
Epoch 148
Validation binary_cross_entropy = 0.793950
Epoch 149
Loss = 2.4314e-02, PNorm = 32.5816, GNorm = 2.1864, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.794001
Epoch 150
Validation binary_cross_entropy = 0.758973
Epoch 151
Validation binary_cross_entropy = 0.768091
Epoch 152
Validation binary_cross_entropy = 0.793369
Epoch 153
Validation binary_cross_entropy = 0.787289
Epoch 154
Loss = 3.6404e-02, PNorm = 32.6016, GNorm = 0.3625, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.752138
Epoch 155
Validation binary_cross_entropy = 0.724064
Epoch 156
Validation binary_cross_entropy = 0.707675
Epoch 157
Validation binary_cross_entropy = 0.678878
Epoch 158
Validation binary_cross_entropy = 0.684026
Epoch 159
Loss = 1.2617e-02, PNorm = 32.6177, GNorm = 0.2241, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.683731
Epoch 160
Validation binary_cross_entropy = 0.668238
Epoch 161
Validation binary_cross_entropy = 0.674242
Epoch 162
Validation binary_cross_entropy = 0.718275
Epoch 163
Validation binary_cross_entropy = 0.765400
Epoch 164
Loss = 3.2618e-02, PNorm = 32.6342, GNorm = 2.8335, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.799893
Epoch 165
Validation binary_cross_entropy = 0.751363
Epoch 166
Validation binary_cross_entropy = 0.707021
Epoch 167
Validation binary_cross_entropy = 0.695971
Epoch 168
Validation binary_cross_entropy = 0.678233
Epoch 169
Loss = 1.3153e-02, PNorm = 32.6530, GNorm = 1.6243, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.670400
Epoch 170
Validation binary_cross_entropy = 0.639893
Epoch 171
Validation binary_cross_entropy = 0.626848
Epoch 172
Validation binary_cross_entropy = 0.724180
Epoch 173
Validation binary_cross_entropy = 0.828024
Epoch 174
Loss = 4.3849e-02, PNorm = 32.6750, GNorm = 0.8221, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.872523
Epoch 175
Validation binary_cross_entropy = 0.876833
Epoch 176
Validation binary_cross_entropy = 0.836393
Epoch 177
Validation binary_cross_entropy = 0.763485
Epoch 178
Validation binary_cross_entropy = 0.698008
Epoch 179
Loss = 8.9685e-02, PNorm = 32.6960, GNorm = 3.5712, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.682533
Epoch 180
Validation binary_cross_entropy = 0.703779
Epoch 181
Validation binary_cross_entropy = 0.746840
Epoch 182
Validation binary_cross_entropy = 0.797879
Epoch 183
Validation binary_cross_entropy = 0.770534
Epoch 184
Loss = 7.4737e-03, PNorm = 32.7139, GNorm = 0.1289, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.743528
Epoch 185
Validation binary_cross_entropy = 0.738652
Epoch 186
Validation binary_cross_entropy = 0.751985
Epoch 187
Validation binary_cross_entropy = 0.735798
Epoch 188
Validation binary_cross_entropy = 0.701948
Epoch 189
Loss = 4.7210e-03, PNorm = 32.7278, GNorm = 0.3498, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.685346
Epoch 190
Validation binary_cross_entropy = 0.670621
Epoch 191
Validation binary_cross_entropy = 0.679786
Epoch 192
Validation binary_cross_entropy = 0.692144
Epoch 193
Validation binary_cross_entropy = 0.709056
Epoch 194
Loss = 2.6896e-02, PNorm = 32.7420, GNorm = 0.0332, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.750141
Epoch 195
Validation binary_cross_entropy = 0.821843
Epoch 196
Validation binary_cross_entropy = 0.894912
Epoch 197
Validation binary_cross_entropy = 0.950115
Epoch 198
Validation binary_cross_entropy = 0.947910
Epoch 199
Loss = 6.7036e-02, PNorm = 32.7604, GNorm = 3.9683, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.902504
Epoch 200
Validation binary_cross_entropy = 0.832340
Epoch 201
Validation binary_cross_entropy = 0.798036
Epoch 202
Validation binary_cross_entropy = 0.786381
Epoch 203
Validation binary_cross_entropy = 0.781401
Epoch 204
Loss = 1.2383e-02, PNorm = 32.7859, GNorm = 0.0528, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.783708
Epoch 205
Validation binary_cross_entropy = 0.805575
Epoch 206
Validation binary_cross_entropy = 0.839593
Epoch 207
Validation binary_cross_entropy = 0.814210
Epoch 208
Validation binary_cross_entropy = 0.780602
Epoch 209
Loss = 1.1270e-02, PNorm = 32.8103, GNorm = 0.2597, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.737878
Epoch 210
Validation binary_cross_entropy = 0.715356
Epoch 211
Validation binary_cross_entropy = 0.718499
Epoch 212
Validation binary_cross_entropy = 0.720584
Epoch 213
Validation binary_cross_entropy = 0.767157
Epoch 214
Loss = 6.8016e-02, PNorm = 32.8381, GNorm = 2.1366, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.776406
Epoch 215
Validation binary_cross_entropy = 0.733183
Epoch 216
Validation binary_cross_entropy = 0.708379
Epoch 217
Validation binary_cross_entropy = 0.701646
Epoch 218
Validation binary_cross_entropy = 0.707178
Epoch 219
Loss = 9.2453e-03, PNorm = 32.8608, GNorm = 0.5483, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.734628
Epoch 220
Validation binary_cross_entropy = 0.752597
Epoch 221
Validation binary_cross_entropy = 0.772915
Epoch 222
Validation binary_cross_entropy = 0.778959
Epoch 223
Validation binary_cross_entropy = 0.761675
Epoch 224
Loss = 1.2570e-02, PNorm = 32.8787, GNorm = 1.3211, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.758330
Epoch 225
Validation binary_cross_entropy = 0.782523
Epoch 226
Validation binary_cross_entropy = 0.744426
Epoch 227
Validation binary_cross_entropy = 0.695959
Epoch 228
Validation binary_cross_entropy = 0.644746
Epoch 229
Loss = 2.9441e-03, PNorm = 32.8967, GNorm = 0.1771, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.624196
Epoch 230
Validation binary_cross_entropy = 0.632178
Epoch 231
Validation binary_cross_entropy = 0.643162
Epoch 232
Validation binary_cross_entropy = 0.664615
Epoch 233
Validation binary_cross_entropy = 0.686167
Epoch 234
Loss = 6.2305e-03, PNorm = 32.9131, GNorm = 0.6431, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.719582
Epoch 235
Validation binary_cross_entropy = 0.735297
Epoch 236
Validation binary_cross_entropy = 0.741883
Epoch 237
Validation binary_cross_entropy = 0.737302
Epoch 238
Validation binary_cross_entropy = 0.740776
Epoch 239
Loss = 1.1489e-01, PNorm = 32.9278, GNorm = 3.7035, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.739519
Epoch 240
Validation binary_cross_entropy = 0.736449
Epoch 241
Validation binary_cross_entropy = 0.712643
Epoch 242
Validation binary_cross_entropy = 0.686670
Epoch 243
Validation binary_cross_entropy = 0.669120
Epoch 244
Loss = 5.3391e-03, PNorm = 32.9397, GNorm = 0.6178, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.659066
Epoch 245
Validation binary_cross_entropy = 0.655926
Epoch 246
Validation binary_cross_entropy = 0.660186
Epoch 247
Validation binary_cross_entropy = 0.667181
Epoch 248
Validation binary_cross_entropy = 0.679373
Epoch 249
Loss = 6.5932e-03, PNorm = 32.9486, GNorm = 0.4014, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.693709
Epoch 250
Validation binary_cross_entropy = 0.709302
Epoch 251
Validation binary_cross_entropy = 0.720456
Epoch 252
Validation binary_cross_entropy = 0.728636
Epoch 253
Validation binary_cross_entropy = 0.734833
Epoch 254
Loss = 6.1730e-03, PNorm = 32.9547, GNorm = 0.0821, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.728806
Epoch 255
Validation binary_cross_entropy = 0.724593
Epoch 256
Validation binary_cross_entropy = 0.713138
Epoch 257
Validation binary_cross_entropy = 0.699924
Epoch 258
Validation binary_cross_entropy = 0.699073
Epoch 259
Loss = 2.0035e-03, PNorm = 32.9603, GNorm = 0.0209, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.703766
Epoch 260
Validation binary_cross_entropy = 0.716406
Epoch 261
Validation binary_cross_entropy = 0.736153
Epoch 262
Validation binary_cross_entropy = 0.756124
Epoch 263
Validation binary_cross_entropy = 0.770229
Epoch 264
Loss = 1.4523e-02, PNorm = 32.9656, GNorm = 0.2824, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.791228
Epoch 265
Validation binary_cross_entropy = 0.805884
Epoch 266
Validation binary_cross_entropy = 0.822688
Epoch 267
Validation binary_cross_entropy = 0.826707
Epoch 268
Validation binary_cross_entropy = 0.816937
Epoch 269
Loss = 1.2948e-02, PNorm = 32.9699, GNorm = 1.6436, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.797235
Epoch 270
Validation binary_cross_entropy = 0.769867
Epoch 271
Validation binary_cross_entropy = 0.742613
Epoch 272
Validation binary_cross_entropy = 0.738689
Epoch 273
Validation binary_cross_entropy = 0.747016
Epoch 274
Loss = 3.4985e-03, PNorm = 32.9760, GNorm = 0.1183, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.760322
Epoch 275
Validation binary_cross_entropy = 0.774247
Epoch 276
Validation binary_cross_entropy = 0.786459
Epoch 277
Validation binary_cross_entropy = 0.803577
Epoch 278
Validation binary_cross_entropy = 0.822741
Epoch 279
Loss = 6.9731e-03, PNorm = 32.9811, GNorm = 0.5857, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.827315
Epoch 280
Validation binary_cross_entropy = 0.817556
Epoch 281
Validation binary_cross_entropy = 0.806473
Epoch 282
Validation binary_cross_entropy = 0.792325
Epoch 283
Validation binary_cross_entropy = 0.777100
Epoch 284
Loss = 7.9104e-03, PNorm = 32.9866, GNorm = 1.0080, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.763542
Epoch 285
Validation binary_cross_entropy = 0.753077
Epoch 286
Validation binary_cross_entropy = 0.747627
Epoch 287
Validation binary_cross_entropy = 0.746756
Epoch 288
Validation binary_cross_entropy = 0.750624
Epoch 289
Loss = 9.9267e-03, PNorm = 32.9929, GNorm = 1.5408, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.763372
Epoch 290
Validation binary_cross_entropy = 0.777148
Epoch 291
Validation binary_cross_entropy = 0.788389
Epoch 292
Validation binary_cross_entropy = 0.803028
Epoch 293
Validation binary_cross_entropy = 0.822294
Epoch 294
Loss = 5.2682e-03, PNorm = 32.9975, GNorm = 0.3020, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.828205
Epoch 295
Validation binary_cross_entropy = 0.816542
Epoch 296
Validation binary_cross_entropy = 0.803713
Epoch 297
Validation binary_cross_entropy = 0.790458
Epoch 298
Validation binary_cross_entropy = 0.776575
Epoch 299
Loss = 1.6102e-03, PNorm = 33.0009, GNorm = 0.0340, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.765406
Model 0 best validation binary_cross_entropy = 0.307598 on epoch 4
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.214004
Ensemble test binary_cross_entropy = 0.214004
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.320598
Epoch 1
Validation binary_cross_entropy = 0.307676
Epoch 2
Validation binary_cross_entropy = 0.843042
Epoch 3
Validation binary_cross_entropy = 0.406598
Epoch 4
Loss = 4.9372e-01, PNorm = 31.6245, GNorm = 9.4774, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.311528
Epoch 5
Validation binary_cross_entropy = 0.519863
Epoch 6
Validation binary_cross_entropy = 0.495987
Epoch 7
Validation binary_cross_entropy = 0.341162
Epoch 8
Validation binary_cross_entropy = 0.356324
Epoch 9
Loss = 3.4000e-01, PNorm = 31.6797, GNorm = 5.1366, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.493001
Epoch 10
Validation binary_cross_entropy = 0.445071
Epoch 11
Validation binary_cross_entropy = 0.361327
Epoch 12
Validation binary_cross_entropy = 0.648680
Epoch 13
Validation binary_cross_entropy = 0.391169
Epoch 14
Loss = 3.5270e-01, PNorm = 31.7527, GNorm = 5.6409, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.424607
Epoch 15
Validation binary_cross_entropy = 0.505982
Epoch 16
Validation binary_cross_entropy = 0.688638
Epoch 17
Validation binary_cross_entropy = 0.443885
Epoch 18
Validation binary_cross_entropy = 0.434603
Epoch 19
Loss = 1.7848e-01, PNorm = 31.8200, GNorm = 5.7006, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.464547
Epoch 20
Validation binary_cross_entropy = 0.555961
Epoch 21
Validation binary_cross_entropy = 0.416275
Epoch 22
Validation binary_cross_entropy = 0.420664
Epoch 23
Validation binary_cross_entropy = 0.468183
Epoch 24
Loss = 1.8310e-01, PNorm = 31.8778, GNorm = 2.7473, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.512584
Epoch 25
Validation binary_cross_entropy = 0.520874
Epoch 26
Validation binary_cross_entropy = 0.521105
Epoch 27
Validation binary_cross_entropy = 0.509808
Epoch 28
Validation binary_cross_entropy = 0.524678
Epoch 29
Loss = 1.7873e-01, PNorm = 31.9256, GNorm = 3.0640, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.523314
Epoch 30
Validation binary_cross_entropy = 0.508813
Epoch 31
Validation binary_cross_entropy = 0.508138
Epoch 32
Validation binary_cross_entropy = 0.488207
Epoch 33
Validation binary_cross_entropy = 0.467923
Epoch 34
Loss = 7.5305e-02, PNorm = 31.9678, GNorm = 1.4653, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.481631
Epoch 35
Validation binary_cross_entropy = 0.462055
Epoch 36
Validation binary_cross_entropy = 0.471250
Epoch 37
Validation binary_cross_entropy = 0.496178
Epoch 38
Validation binary_cross_entropy = 0.522755
Epoch 39
Loss = 1.3775e-01, PNorm = 32.0065, GNorm = 3.0493, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.532594
Epoch 40
Validation binary_cross_entropy = 0.468297
Epoch 41
Validation binary_cross_entropy = 0.438477
Epoch 42
Validation binary_cross_entropy = 0.452725
Epoch 43
Validation binary_cross_entropy = 0.504260
Epoch 44
Loss = 1.1359e-01, PNorm = 32.0462, GNorm = 1.7221, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.540652
Epoch 45
Validation binary_cross_entropy = 0.516347
Epoch 46
Validation binary_cross_entropy = 0.532113
Epoch 47
Validation binary_cross_entropy = 0.516257
Epoch 48
Validation binary_cross_entropy = 0.536933
Epoch 49
Loss = 4.2769e-02, PNorm = 32.0828, GNorm = 1.1849, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.549247
Epoch 50
Validation binary_cross_entropy = 0.534938
Epoch 51
Validation binary_cross_entropy = 0.567290
Epoch 52
Validation binary_cross_entropy = 0.565520
Epoch 53
Validation binary_cross_entropy = 0.639797
Epoch 54
Loss = 1.0532e-01, PNorm = 32.1168, GNorm = 2.9424, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.594887
Epoch 55
Validation binary_cross_entropy = 0.541953
Epoch 56
Validation binary_cross_entropy = 0.535720
Epoch 57
Validation binary_cross_entropy = 0.531922
Epoch 58
Validation binary_cross_entropy = 0.570801
Epoch 59
Loss = 1.1389e-01, PNorm = 32.1478, GNorm = 2.0471, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.574345
Epoch 60
Validation binary_cross_entropy = 0.545531
Epoch 61
Validation binary_cross_entropy = 0.557892
Epoch 62
Validation binary_cross_entropy = 0.638219
Epoch 63
Validation binary_cross_entropy = 0.646645
Epoch 64
Loss = 1.8551e-01, PNorm = 32.1774, GNorm = 3.4598, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.521098
Epoch 65
Validation binary_cross_entropy = 0.530756
Epoch 66
Validation binary_cross_entropy = 0.570761
Epoch 67
Validation binary_cross_entropy = 0.590417
Epoch 68
Validation binary_cross_entropy = 0.577289
Epoch 69
Loss = 3.9868e-02, PNorm = 32.2112, GNorm = 1.4272, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.568768
Epoch 70
Validation binary_cross_entropy = 0.548365
Epoch 71
Validation binary_cross_entropy = 0.571720
Epoch 72
Validation binary_cross_entropy = 0.569908
Epoch 73
Validation binary_cross_entropy = 0.550633
Epoch 74
Loss = 5.4792e-02, PNorm = 32.2460, GNorm = 2.0277, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.528898
Epoch 75
Validation binary_cross_entropy = 0.535946
Epoch 76
Validation binary_cross_entropy = 0.541815
Epoch 77
Validation binary_cross_entropy = 0.541457
Epoch 78
Validation binary_cross_entropy = 0.579643
Epoch 79
Loss = 8.4907e-02, PNorm = 32.2748, GNorm = 3.3876, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.649616
Epoch 80
Validation binary_cross_entropy = 0.555816
Epoch 81
Validation binary_cross_entropy = 0.505635
Epoch 82
Validation binary_cross_entropy = 0.503354
Epoch 83
Validation binary_cross_entropy = 0.522356
Epoch 84
Loss = 2.3978e-02, PNorm = 32.3108, GNorm = 1.9064, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.581224
Epoch 85
Validation binary_cross_entropy = 0.618555
Epoch 86
Validation binary_cross_entropy = 0.614687
Epoch 87
Validation binary_cross_entropy = 0.612071
Epoch 88
Validation binary_cross_entropy = 0.615851
Epoch 89
Loss = 2.5663e-02, PNorm = 32.3432, GNorm = 2.5269, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.630694
Epoch 90
Validation binary_cross_entropy = 0.699154
Epoch 91
Validation binary_cross_entropy = 0.747861
Epoch 92
Validation binary_cross_entropy = 0.733656
Epoch 93
Validation binary_cross_entropy = 0.673404
Epoch 94
Loss = 1.1285e-02, PNorm = 32.3697, GNorm = 0.6794, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.660005
Epoch 95
Validation binary_cross_entropy = 0.700314
Epoch 96
Validation binary_cross_entropy = 0.741448
Epoch 97
Validation binary_cross_entropy = 0.698063
Epoch 98
Validation binary_cross_entropy = 0.674911
Epoch 99
Loss = 1.3201e-02, PNorm = 32.3917, GNorm = 0.2959, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.682359
Epoch 100
Validation binary_cross_entropy = 0.669918
Epoch 101
Validation binary_cross_entropy = 0.658227
Epoch 102
Validation binary_cross_entropy = 0.635378
Epoch 103
Validation binary_cross_entropy = 0.634721
Epoch 104
Loss = 2.3890e-02, PNorm = 32.4126, GNorm = 1.2262, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.664065
Epoch 105
Validation binary_cross_entropy = 0.676155
Epoch 106
Validation binary_cross_entropy = 0.671257
Epoch 107
Validation binary_cross_entropy = 0.678131
Epoch 108
Validation binary_cross_entropy = 0.674708
Epoch 109
Loss = 7.7509e-03, PNorm = 32.4346, GNorm = 0.5781, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.690117
Epoch 110
Validation binary_cross_entropy = 0.718302
Epoch 111
Validation binary_cross_entropy = 0.726640
Epoch 112
Validation binary_cross_entropy = 0.733379
Epoch 113
Validation binary_cross_entropy = 0.747050
Epoch 114
Loss = 3.1149e-02, PNorm = 32.4550, GNorm = 2.3858, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.776564
Epoch 115
Validation binary_cross_entropy = 0.734712
Epoch 116
Validation binary_cross_entropy = 0.671049
Epoch 117
Validation binary_cross_entropy = 0.651287
Epoch 118
Validation binary_cross_entropy = 0.691317
Epoch 119
Loss = 1.1937e-02, PNorm = 32.4741, GNorm = 0.6597, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.743954
Epoch 120
Validation binary_cross_entropy = 0.790835
Epoch 121
Validation binary_cross_entropy = 0.761844
Epoch 122
Validation binary_cross_entropy = 0.712532
Epoch 123
Validation binary_cross_entropy = 0.705460
Epoch 124
Loss = 1.2173e-01, PNorm = 32.4979, GNorm = 3.7731, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.720510
Epoch 125
Validation binary_cross_entropy = 0.796592
Epoch 126
Validation binary_cross_entropy = 0.678274
Epoch 127
Validation binary_cross_entropy = 0.564373
Epoch 128
Validation binary_cross_entropy = 0.566027
Epoch 129
Loss = 5.3214e-02, PNorm = 32.5266, GNorm = 1.7257, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.603778
Epoch 130
Validation binary_cross_entropy = 0.672433
Epoch 131
Validation binary_cross_entropy = 0.674985
Epoch 132
Validation binary_cross_entropy = 0.685572
Epoch 133
Validation binary_cross_entropy = 0.715427
Epoch 134
Loss = 4.9374e-02, PNorm = 32.5614, GNorm = 2.0766, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.692737
Epoch 135
Validation binary_cross_entropy = 0.658967
Epoch 136
Validation binary_cross_entropy = 0.631831
Epoch 137
Validation binary_cross_entropy = 0.605539
Epoch 138
Validation binary_cross_entropy = 0.603334
Epoch 139
Loss = 1.6974e-02, PNorm = 32.5921, GNorm = 0.9811, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.626529
Epoch 140
Validation binary_cross_entropy = 0.629484
Epoch 141
Validation binary_cross_entropy = 0.601130
Epoch 142
Validation binary_cross_entropy = 0.586339
Epoch 143
Validation binary_cross_entropy = 0.585549
Epoch 144
Loss = 2.3108e-02, PNorm = 32.6188, GNorm = 1.5453, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.592101
Epoch 145
Validation binary_cross_entropy = 0.599821
Epoch 146
Validation binary_cross_entropy = 0.633636
Epoch 147
Validation binary_cross_entropy = 0.689665
Epoch 148
Validation binary_cross_entropy = 0.687767
Epoch 149
Loss = 5.0917e-02, PNorm = 32.6389, GNorm = 1.0697, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.658507
Epoch 150
Validation binary_cross_entropy = 0.639628
Epoch 151
Validation binary_cross_entropy = 0.635473
Epoch 152
Validation binary_cross_entropy = 0.656883
Epoch 153
Validation binary_cross_entropy = 0.748149
Epoch 154
Loss = 8.4211e-03, PNorm = 32.6564, GNorm = 0.2863, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.863433
Epoch 155
Validation binary_cross_entropy = 0.821077
Epoch 156
Validation binary_cross_entropy = 0.723544
Epoch 157
Validation binary_cross_entropy = 0.681383
Epoch 158
Validation binary_cross_entropy = 0.675683
Epoch 159
Loss = 1.3919e-02, PNorm = 32.6793, GNorm = 1.0846, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.668006
Epoch 160
Validation binary_cross_entropy = 0.671951
Epoch 161
Validation binary_cross_entropy = 0.710794
Epoch 162
Validation binary_cross_entropy = 0.767939
Epoch 163
Validation binary_cross_entropy = 0.761445
Epoch 164
Loss = 1.0623e-02, PNorm = 32.6993, GNorm = 1.2357, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.741638
Epoch 165
Validation binary_cross_entropy = 0.703723
Epoch 166
Validation binary_cross_entropy = 0.702969
Epoch 167
Validation binary_cross_entropy = 0.719178
Epoch 168
Validation binary_cross_entropy = 0.744705
Epoch 169
Loss = 5.2656e-03, PNorm = 32.7148, GNorm = 0.2257, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.771772
Epoch 170
Validation binary_cross_entropy = 0.809885
Epoch 171
Validation binary_cross_entropy = 0.832111
Epoch 172
Validation binary_cross_entropy = 0.832435
Epoch 173
Validation binary_cross_entropy = 0.813470
Epoch 174
Loss = 2.7874e-02, PNorm = 32.7331, GNorm = 1.7198, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.786483
Epoch 175
Validation binary_cross_entropy = 0.785623
Epoch 176
Validation binary_cross_entropy = 0.782572
Epoch 177
Validation binary_cross_entropy = 0.785907
Epoch 178
Validation binary_cross_entropy = 0.790730
Epoch 179
Loss = 4.0351e-03, PNorm = 32.7524, GNorm = 0.1876, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.790137
Epoch 180
Validation binary_cross_entropy = 0.807757
Epoch 181
Validation binary_cross_entropy = 0.814752
Epoch 182
Validation binary_cross_entropy = 0.798382
Epoch 183
Validation binary_cross_entropy = 0.777269
Epoch 184
Loss = 5.9149e-02, PNorm = 32.7671, GNorm = 0.6606, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.738961
Epoch 185
Validation binary_cross_entropy = 0.702888
Epoch 186
Validation binary_cross_entropy = 0.688278
Epoch 187
Validation binary_cross_entropy = 0.684877
Epoch 188
Validation binary_cross_entropy = 0.692576
Epoch 189
Loss = 7.9231e-03, PNorm = 32.7840, GNorm = 0.8415, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.712485
Epoch 190
Validation binary_cross_entropy = 0.738889
Epoch 191
Validation binary_cross_entropy = 0.759537
Epoch 192
Validation binary_cross_entropy = 0.762667
Epoch 193
Validation binary_cross_entropy = 0.766839
Epoch 194
Loss = 2.3621e-03, PNorm = 32.7980, GNorm = 0.0348, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.770596
Epoch 195
Validation binary_cross_entropy = 0.778072
Epoch 196
Validation binary_cross_entropy = 0.801422
Epoch 197
Validation binary_cross_entropy = 0.830414
Epoch 198
Validation binary_cross_entropy = 0.828773
Epoch 199
Loss = 1.2790e-02, PNorm = 32.8079, GNorm = 1.8539, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.803158
Epoch 200
Validation binary_cross_entropy = 0.776488
Epoch 201
Validation binary_cross_entropy = 0.778440
Epoch 202
Validation binary_cross_entropy = 0.794234
Epoch 203
Validation binary_cross_entropy = 0.814908
Epoch 204
Loss = 2.5793e-02, PNorm = 32.8169, GNorm = 2.1008, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.781981
Epoch 205
Validation binary_cross_entropy = 0.754635
Epoch 206
Validation binary_cross_entropy = 0.773077
Epoch 207
Validation binary_cross_entropy = 0.792854
Epoch 208
Validation binary_cross_entropy = 0.808777
Epoch 209
Loss = 1.0754e-01, PNorm = 32.8302, GNorm = 0.2629, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.835156
Epoch 210
Validation binary_cross_entropy = 0.860660
Epoch 211
Validation binary_cross_entropy = 0.867928
Epoch 212
Validation binary_cross_entropy = 0.829282
Epoch 213
Validation binary_cross_entropy = 0.797042
Epoch 214
Loss = 4.3293e-03, PNorm = 32.8408, GNorm = 0.5156, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.779572
Epoch 215
Validation binary_cross_entropy = 0.812272
Epoch 216
Validation binary_cross_entropy = 0.863442
Epoch 217
Validation binary_cross_entropy = 0.865086
Epoch 218
Validation binary_cross_entropy = 0.828377
Epoch 219
Loss = 6.7186e-02, PNorm = 32.8535, GNorm = 2.4315, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.793607
Epoch 220
Validation binary_cross_entropy = 0.846156
Epoch 221
Validation binary_cross_entropy = 0.880962
Epoch 222
Validation binary_cross_entropy = 0.843237
Epoch 223
Validation binary_cross_entropy = 0.837998
Epoch 224
Loss = 2.7624e-02, PNorm = 32.8731, GNorm = 0.1507, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.850395
Epoch 225
Validation binary_cross_entropy = 0.853686
Epoch 226
Validation binary_cross_entropy = 0.791273
Epoch 227
Validation binary_cross_entropy = 0.760406
Epoch 228
Validation binary_cross_entropy = 0.769953
Epoch 229
Loss = 1.1666e-02, PNorm = 32.8922, GNorm = 1.4097, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.781753
Epoch 230
Validation binary_cross_entropy = 0.789937
Epoch 231
Validation binary_cross_entropy = 0.808513
Epoch 232
Validation binary_cross_entropy = 0.803476
Epoch 233
Validation binary_cross_entropy = 0.800847
Epoch 234
Loss = 2.7691e-03, PNorm = 32.9082, GNorm = 0.0150, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.799117
Epoch 235
Validation binary_cross_entropy = 0.805435
Epoch 236
Validation binary_cross_entropy = 0.861400
Epoch 237
Validation binary_cross_entropy = 0.906053
Epoch 238
Validation binary_cross_entropy = 0.883926
Epoch 239
Loss = 4.3495e-02, PNorm = 32.9221, GNorm = 2.1873, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.835597
Epoch 240
Validation binary_cross_entropy = 0.814101
Epoch 241
Validation binary_cross_entropy = 0.794231
Epoch 242
Validation binary_cross_entropy = 0.782169
Epoch 243
Validation binary_cross_entropy = 0.779876
Epoch 244
Loss = 3.8685e-02, PNorm = 32.9379, GNorm = 2.6287, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.773297
Epoch 245
Validation binary_cross_entropy = 0.773045
Epoch 246
Validation binary_cross_entropy = 0.772355
Epoch 247
Validation binary_cross_entropy = 0.763693
Epoch 248
Validation binary_cross_entropy = 0.763269
Epoch 249
Loss = 8.5784e-03, PNorm = 32.9508, GNorm = 0.7907, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.776272
Epoch 250
Validation binary_cross_entropy = 0.719791
Epoch 251
Validation binary_cross_entropy = 0.717299
Epoch 252
Validation binary_cross_entropy = 0.735896
Epoch 253
Validation binary_cross_entropy = 0.754161
Epoch 254
Loss = 8.7351e-03, PNorm = 32.9686, GNorm = 0.6554, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.757828
Epoch 255
Validation binary_cross_entropy = 0.751422
Epoch 256
Validation binary_cross_entropy = 0.757961
Epoch 257
Validation binary_cross_entropy = 0.782571
Epoch 258
Validation binary_cross_entropy = 0.832721
Epoch 259
Loss = 4.2438e-02, PNorm = 32.9869, GNorm = 3.2357, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.885769
Epoch 260
Validation binary_cross_entropy = 0.931541
Epoch 261
Validation binary_cross_entropy = 0.949458
Epoch 262
Validation binary_cross_entropy = 0.933818
Epoch 263
Validation binary_cross_entropy = 0.928019
Epoch 264
Loss = 3.3543e-02, PNorm = 33.0015, GNorm = 1.6805, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.890674
Epoch 265
Validation binary_cross_entropy = 0.914748
Epoch 266
Validation binary_cross_entropy = 0.937498
Epoch 267
Validation binary_cross_entropy = 0.923738
Epoch 268
Validation binary_cross_entropy = 0.919111
Epoch 269
Loss = 7.8726e-03, PNorm = 33.0218, GNorm = 0.0955, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.920929
Epoch 270
Validation binary_cross_entropy = 0.940986
Epoch 271
Validation binary_cross_entropy = 0.949708
Epoch 272
Validation binary_cross_entropy = 0.936364
Epoch 273
Validation binary_cross_entropy = 0.925110
Epoch 274
Loss = 2.3393e-02, PNorm = 33.0404, GNorm = 1.0034, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.924582
Epoch 275
Validation binary_cross_entropy = 0.931885
Epoch 276
Validation binary_cross_entropy = 0.913067
Epoch 277
Validation binary_cross_entropy = 0.870184
Epoch 278
Validation binary_cross_entropy = 0.829195
Epoch 279
Loss = 1.0922e-02, PNorm = 33.0605, GNorm = 0.0918, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.799873
Epoch 280
Validation binary_cross_entropy = 0.789355
Epoch 281
Validation binary_cross_entropy = 0.789500
Epoch 282
Validation binary_cross_entropy = 0.797903
Epoch 283
Validation binary_cross_entropy = 0.826829
Epoch 284
Loss = 2.3964e-03, PNorm = 33.0749, GNorm = 0.1154, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.854981
Epoch 285
Validation binary_cross_entropy = 0.879995
Epoch 286
Validation binary_cross_entropy = 0.871434
Epoch 287
Validation binary_cross_entropy = 0.862273
Epoch 288
Validation binary_cross_entropy = 0.854629
Epoch 289
Loss = 1.1683e-02, PNorm = 33.0829, GNorm = 1.1623, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.857438
Epoch 290
Validation binary_cross_entropy = 0.883638
Epoch 291
Validation binary_cross_entropy = 0.919874
Epoch 292
Validation binary_cross_entropy = 0.961637
Epoch 293
Validation binary_cross_entropy = 0.993073
Epoch 294
Loss = 9.7092e-03, PNorm = 33.0888, GNorm = 0.9531, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.992622
Epoch 295
Validation binary_cross_entropy = 0.975151
Epoch 296
Validation binary_cross_entropy = 0.954637
Epoch 297
Validation binary_cross_entropy = 0.921660
Epoch 298
Validation binary_cross_entropy = 0.889526
Epoch 299
Loss = 5.3835e-03, PNorm = 33.0969, GNorm = 0.0584, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.858002
Model 0 best validation binary_cross_entropy = 0.307676 on epoch 1
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.236231
Ensemble test binary_cross_entropy = 0.236231
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.305001
Epoch 1
Validation binary_cross_entropy = 0.316630
Epoch 2
Validation binary_cross_entropy = 1.142876
Epoch 3
Validation binary_cross_entropy = 0.673778
Epoch 4
Loss = 6.8329e-01, PNorm = 31.6244, GNorm = 7.4911, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297519
Epoch 5
Validation binary_cross_entropy = 0.377887
Epoch 6
Validation binary_cross_entropy = 0.604069
Epoch 7
Validation binary_cross_entropy = 0.413691
Epoch 8
Validation binary_cross_entropy = 0.328656
Epoch 9
Loss = 4.3132e-01, PNorm = 31.6789, GNorm = 7.5577, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.386833
Epoch 10
Validation binary_cross_entropy = 0.483254
Epoch 11
Validation binary_cross_entropy = 0.342499
Epoch 12
Validation binary_cross_entropy = 0.349767
Epoch 13
Validation binary_cross_entropy = 0.433764
Epoch 14
Loss = 1.6699e-01, PNorm = 31.7488, GNorm = 2.6174, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.484171
Epoch 15
Validation binary_cross_entropy = 0.502510
Epoch 16
Validation binary_cross_entropy = 0.459767
Epoch 17
Validation binary_cross_entropy = 0.447285
Epoch 18
Validation binary_cross_entropy = 0.466082
Epoch 19
Loss = 1.7851e-01, PNorm = 31.8088, GNorm = 2.0695, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.555034
Epoch 20
Validation binary_cross_entropy = 0.503496
Epoch 21
Validation binary_cross_entropy = 0.419541
Epoch 22
Validation binary_cross_entropy = 0.422534
Epoch 23
Validation binary_cross_entropy = 0.517039
Epoch 24
Loss = 2.0485e-01, PNorm = 31.8616, GNorm = 3.2717, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.471236
Epoch 25
Validation binary_cross_entropy = 0.476467
Epoch 26
Validation binary_cross_entropy = 0.483336
Epoch 27
Validation binary_cross_entropy = 0.489155
Epoch 28
Validation binary_cross_entropy = 0.438966
Epoch 29
Loss = 2.2638e-01, PNorm = 31.9100, GNorm = 1.9943, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.440900
Epoch 30
Validation binary_cross_entropy = 0.457290
Epoch 31
Validation binary_cross_entropy = 0.467023
Epoch 32
Validation binary_cross_entropy = 0.553632
Epoch 33
Validation binary_cross_entropy = 0.610411
Epoch 34
Loss = 1.3765e-01, PNorm = 31.9565, GNorm = 3.7021, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.618442
Epoch 35
Validation binary_cross_entropy = 0.571013
Epoch 36
Validation binary_cross_entropy = 0.563458
Epoch 37
Validation binary_cross_entropy = 0.509189
Epoch 38
Validation binary_cross_entropy = 0.445166
Epoch 39
Loss = 6.0391e-02, PNorm = 31.9965, GNorm = 1.6559, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.423308
Epoch 40
Validation binary_cross_entropy = 0.448267
Epoch 41
Validation binary_cross_entropy = 0.426649
Epoch 42
Validation binary_cross_entropy = 0.435406
Epoch 43
Validation binary_cross_entropy = 0.417656
Epoch 44
Loss = 7.7322e-02, PNorm = 32.0324, GNorm = 1.9695, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.427839
Epoch 45
Validation binary_cross_entropy = 0.393648
Epoch 46
Validation binary_cross_entropy = 0.398882
Epoch 47
Validation binary_cross_entropy = 0.480305
Epoch 48
Validation binary_cross_entropy = 0.469092
Epoch 49
Loss = 9.2229e-02, PNorm = 32.0669, GNorm = 1.3406, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.465569
Epoch 50
Validation binary_cross_entropy = 0.526204
Epoch 51
Validation binary_cross_entropy = 0.490806
Epoch 52
Validation binary_cross_entropy = 0.540987
Epoch 53
Validation binary_cross_entropy = 0.543423
Epoch 54
Loss = 7.9348e-02, PNorm = 32.1058, GNorm = 1.7293, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.535693
Epoch 55
Validation binary_cross_entropy = 0.576008
Epoch 56
Validation binary_cross_entropy = 0.562208
Epoch 57
Validation binary_cross_entropy = 0.564972
Epoch 58
Validation binary_cross_entropy = 0.594559
Epoch 59
Loss = 1.0989e-01, PNorm = 32.1466, GNorm = 3.8931, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.539757
Epoch 60
Validation binary_cross_entropy = 0.535639
Epoch 61
Validation binary_cross_entropy = 0.561490
Epoch 62
Validation binary_cross_entropy = 0.579617
Epoch 63
Validation binary_cross_entropy = 0.605004
Epoch 64
Loss = 5.9311e-02, PNorm = 32.1842, GNorm = 1.4949, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.598791
Epoch 65
Validation binary_cross_entropy = 0.561628
Epoch 66
Validation binary_cross_entropy = 0.528119
Epoch 67
Validation binary_cross_entropy = 0.501559
Epoch 68
Validation binary_cross_entropy = 0.505175
Epoch 69
Loss = 3.2637e-02, PNorm = 32.2183, GNorm = 0.8089, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.504600
Epoch 70
Validation binary_cross_entropy = 0.521161
Epoch 71
Validation binary_cross_entropy = 0.537172
Epoch 72
Validation binary_cross_entropy = 0.535504
Epoch 73
Validation binary_cross_entropy = 0.548591
Epoch 74
Loss = 4.2042e-02, PNorm = 32.2471, GNorm = 2.0675, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.591816
Epoch 75
Validation binary_cross_entropy = 0.622391
Epoch 76
Validation binary_cross_entropy = 0.633696
Epoch 77
Validation binary_cross_entropy = 0.652161
Epoch 78
Validation binary_cross_entropy = 0.615878
Epoch 79
Loss = 9.5160e-02, PNorm = 32.2730, GNorm = 0.9738, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.555273
Epoch 80
Validation binary_cross_entropy = 0.551255
Epoch 81
Validation binary_cross_entropy = 0.548799
Epoch 82
Validation binary_cross_entropy = 0.648329
Epoch 83
Validation binary_cross_entropy = 0.622305
Epoch 84
Loss = 8.2160e-02, PNorm = 32.2983, GNorm = 2.7787, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.550281
Epoch 85
Validation binary_cross_entropy = 0.568339
Epoch 86
Validation binary_cross_entropy = 0.592280
Epoch 87
Validation binary_cross_entropy = 0.546877
Epoch 88
Validation binary_cross_entropy = 0.539496
Epoch 89
Loss = 3.6130e-02, PNorm = 32.3269, GNorm = 1.6088, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.621467
Epoch 90
Validation binary_cross_entropy = 0.628875
Epoch 91
Validation binary_cross_entropy = 0.579148
Epoch 92
Validation binary_cross_entropy = 0.575253
Epoch 93
Validation binary_cross_entropy = 0.590293
Epoch 94
Loss = 1.9569e-02, PNorm = 32.3532, GNorm = 0.9732, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.590238
Epoch 95
Validation binary_cross_entropy = 0.576936
Epoch 96
Validation binary_cross_entropy = 0.597250
Epoch 97
Validation binary_cross_entropy = 0.647768
Epoch 98
Validation binary_cross_entropy = 0.656990
Epoch 99
Loss = 5.4628e-02, PNorm = 32.3776, GNorm = 0.6570, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.636673
Epoch 100
Validation binary_cross_entropy = 0.684352
Epoch 101
Validation binary_cross_entropy = 0.758168
Epoch 102
Validation binary_cross_entropy = 0.717478
Epoch 103
Validation binary_cross_entropy = 0.695382
Epoch 104
Loss = 3.7763e-02, PNorm = 32.4019, GNorm = 2.4926, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.702866
Epoch 105
Validation binary_cross_entropy = 0.658919
Epoch 106
Validation binary_cross_entropy = 0.613281
Epoch 107
Validation binary_cross_entropy = 0.609663
Epoch 108
Validation binary_cross_entropy = 0.616982
Epoch 109
Loss = 4.6984e-02, PNorm = 32.4240, GNorm = 2.4000, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.636330
Epoch 110
Validation binary_cross_entropy = 0.677965
Epoch 111
Validation binary_cross_entropy = 0.723224
Epoch 112
Validation binary_cross_entropy = 0.723462
Epoch 113
Validation binary_cross_entropy = 0.659501
Epoch 114
Loss = 2.1503e-02, PNorm = 32.4488, GNorm = 1.2163, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.637143
Epoch 115
Validation binary_cross_entropy = 0.661195
Epoch 116
Validation binary_cross_entropy = 0.669808
Epoch 117
Validation binary_cross_entropy = 0.675317
Epoch 118
Validation binary_cross_entropy = 0.708751
Epoch 119
Loss = 6.9494e-02, PNorm = 32.4746, GNorm = 3.3657, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.713710
Epoch 120
Validation binary_cross_entropy = 0.689934
Epoch 121
Validation binary_cross_entropy = 0.676996
Epoch 122
Validation binary_cross_entropy = 0.664905
Epoch 123
Validation binary_cross_entropy = 0.654867
Epoch 124
Loss = 3.2431e-02, PNorm = 32.4970, GNorm = 1.6812, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.637054
Epoch 125
Validation binary_cross_entropy = 0.628922
Epoch 126
Validation binary_cross_entropy = 0.635587
Epoch 127
Validation binary_cross_entropy = 0.662757
Epoch 128
Validation binary_cross_entropy = 0.686760
Epoch 129
Loss = 2.5275e-02, PNorm = 32.5144, GNorm = 2.4602, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.702349
Epoch 130
Validation binary_cross_entropy = 0.687747
Epoch 131
Validation binary_cross_entropy = 0.682460
Epoch 132
Validation binary_cross_entropy = 0.691406
Epoch 133
Validation binary_cross_entropy = 0.690468
Epoch 134
Loss = 7.7160e-03, PNorm = 32.5327, GNorm = 0.2574, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.685271
Epoch 135
Validation binary_cross_entropy = 0.680961
Epoch 136
Validation binary_cross_entropy = 0.691458
Epoch 137
Validation binary_cross_entropy = 0.710559
Epoch 138
Validation binary_cross_entropy = 0.733760
Epoch 139
Loss = 1.1671e-02, PNorm = 32.5488, GNorm = 0.7007, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.749051
Epoch 140
Validation binary_cross_entropy = 0.753558
Epoch 141
Validation binary_cross_entropy = 0.751988
Epoch 142
Validation binary_cross_entropy = 0.748870
Epoch 143
Validation binary_cross_entropy = 0.747088
Epoch 144
Loss = 2.9862e-02, PNorm = 32.5628, GNorm = 1.1373, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.752999
Epoch 145
Validation binary_cross_entropy = 0.760673
Epoch 146
Validation binary_cross_entropy = 0.761778
Epoch 147
Validation binary_cross_entropy = 0.764809
Epoch 148
Validation binary_cross_entropy = 0.776373
Epoch 149
Loss = 1.3123e-02, PNorm = 32.5736, GNorm = 0.7777, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.799173
Epoch 150
Validation binary_cross_entropy = 0.823501
Epoch 151
Validation binary_cross_entropy = 0.827926
Epoch 152
Validation binary_cross_entropy = 0.812489
Epoch 153
Validation binary_cross_entropy = 0.807824
Epoch 154
Loss = 1.9797e-02, PNorm = 32.5823, GNorm = 1.6118, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.809071
Epoch 155
Validation binary_cross_entropy = 0.815820
Epoch 156
Validation binary_cross_entropy = 0.844663
Epoch 157
Validation binary_cross_entropy = 0.893881
Epoch 158
Validation binary_cross_entropy = 0.932721
Epoch 159
Loss = 1.0149e-02, PNorm = 32.5922, GNorm = 0.4747, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.937074
Epoch 160
Validation binary_cross_entropy = 0.914581
Epoch 161
Validation binary_cross_entropy = 0.854154
Epoch 162
Validation binary_cross_entropy = 0.788535
Epoch 163
Validation binary_cross_entropy = 0.772706
Epoch 164
Loss = 8.0771e-03, PNorm = 32.6038, GNorm = 0.6969, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.773237
Epoch 165
Validation binary_cross_entropy = 0.780633
Epoch 166
Validation binary_cross_entropy = 0.833930
Epoch 167
Validation binary_cross_entropy = 0.893621
Epoch 168
Validation binary_cross_entropy = 0.912853
Epoch 169
Loss = 1.4304e-02, PNorm = 32.6156, GNorm = 1.6418, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.886065
Epoch 170
Validation binary_cross_entropy = 0.838904
Epoch 171
Validation binary_cross_entropy = 0.821211
Epoch 172
Validation binary_cross_entropy = 0.833797
Epoch 173
Validation binary_cross_entropy = 0.904449
Epoch 174
Loss = 1.1117e-02, PNorm = 32.6297, GNorm = 1.2327, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.972660
Epoch 175
Validation binary_cross_entropy = 0.966573
Epoch 176
Validation binary_cross_entropy = 0.854309
Epoch 177
Validation binary_cross_entropy = 0.799090
Epoch 178
Validation binary_cross_entropy = 0.761245
Epoch 179
Loss = 3.7837e-02, PNorm = 32.6476, GNorm = 0.1109, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.785730
Epoch 180
Validation binary_cross_entropy = 0.842648
Epoch 181
Validation binary_cross_entropy = 0.847815
Epoch 182
Validation binary_cross_entropy = 0.782419
Epoch 183
Validation binary_cross_entropy = 0.739856
Epoch 184
Loss = 7.5234e-02, PNorm = 32.6656, GNorm = 3.3505, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.733478
Epoch 185
Validation binary_cross_entropy = 0.764830
Epoch 186
Validation binary_cross_entropy = 0.839049
Epoch 187
Validation binary_cross_entropy = 0.870915
Epoch 188
Validation binary_cross_entropy = 0.911940
Epoch 189
Loss = 1.0342e-02, PNorm = 32.6851, GNorm = 0.7661, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.934151
Epoch 190
Validation binary_cross_entropy = 0.849045
Epoch 191
Validation binary_cross_entropy = 0.801799
Epoch 192
Validation binary_cross_entropy = 0.780374
Epoch 193
Validation binary_cross_entropy = 0.785695
Epoch 194
Loss = 2.2583e-02, PNorm = 32.7046, GNorm = 0.0816, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.786846
Epoch 195
Validation binary_cross_entropy = 0.797106
Epoch 196
Validation binary_cross_entropy = 0.849344
Epoch 197
Validation binary_cross_entropy = 0.861980
Epoch 198
Validation binary_cross_entropy = 0.845006
Epoch 199
Loss = 1.0836e-02, PNorm = 32.7269, GNorm = 1.0667, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.835524
Epoch 200
Validation binary_cross_entropy = 0.831414
Epoch 201
Validation binary_cross_entropy = 0.818718
Epoch 202
Validation binary_cross_entropy = 0.821497
Epoch 203
Validation binary_cross_entropy = 0.824024
Epoch 204
Loss = 3.4719e-02, PNorm = 32.7433, GNorm = 3.4411, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.873819
Epoch 205
Validation binary_cross_entropy = 0.949346
Epoch 206
Validation binary_cross_entropy = 0.932676
Epoch 207
Validation binary_cross_entropy = 0.827714
Epoch 208
Validation binary_cross_entropy = 0.771793
Epoch 209
Loss = 9.5299e-03, PNorm = 32.7594, GNorm = 0.0902, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.764836
Epoch 210
Validation binary_cross_entropy = 0.771441
Epoch 211
Validation binary_cross_entropy = 0.775897
Epoch 212
Validation binary_cross_entropy = 0.791525
Epoch 213
Validation binary_cross_entropy = 0.812711
Epoch 214
Loss = 1.0438e-02, PNorm = 32.7737, GNorm = 0.0493, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.832740
Epoch 215
Validation binary_cross_entropy = 0.851836
Epoch 216
Validation binary_cross_entropy = 0.852581
Epoch 217
Validation binary_cross_entropy = 0.848723
Epoch 218
Validation binary_cross_entropy = 0.846885
Epoch 219
Loss = 3.7052e-03, PNorm = 32.7833, GNorm = 0.3000, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.845138
Epoch 220
Validation binary_cross_entropy = 0.870197
Epoch 221
Validation binary_cross_entropy = 0.894301
Epoch 222
Validation binary_cross_entropy = 0.913704
Epoch 223
Validation binary_cross_entropy = 0.919910
Epoch 224
Loss = 7.9453e-03, PNorm = 32.7903, GNorm = 0.0657, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.908755
Epoch 225
Validation binary_cross_entropy = 0.904240
Epoch 226
Validation binary_cross_entropy = 0.893091
Epoch 227
Validation binary_cross_entropy = 0.871866
Epoch 228
Validation binary_cross_entropy = 0.890380
Epoch 229
Loss = 1.7488e-03, PNorm = 32.7961, GNorm = 0.1556, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.909342
Epoch 230
Validation binary_cross_entropy = 0.937269
Epoch 231
Validation binary_cross_entropy = 0.953492
Epoch 232
Validation binary_cross_entropy = 0.937358
Epoch 233
Validation binary_cross_entropy = 0.908000
Epoch 234
Loss = 7.6931e-03, PNorm = 32.8036, GNorm = 1.1964, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.879007
Epoch 235
Validation binary_cross_entropy = 0.847481
Epoch 236
Validation binary_cross_entropy = 0.837857
Epoch 237
Validation binary_cross_entropy = 0.830858
Epoch 238
Validation binary_cross_entropy = 0.822061
Epoch 239
Loss = 1.8152e-02, PNorm = 32.8117, GNorm = 2.2689, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.808805
Epoch 240
Validation binary_cross_entropy = 0.798091
Epoch 241
Validation binary_cross_entropy = 0.794185
Epoch 242
Validation binary_cross_entropy = 0.796766
Epoch 243
Validation binary_cross_entropy = 0.813375
Epoch 244
Loss = 7.2463e-03, PNorm = 32.8186, GNorm = 0.7264, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.833671
Epoch 245
Validation binary_cross_entropy = 0.860155
Epoch 246
Validation binary_cross_entropy = 0.919778
Epoch 247
Validation binary_cross_entropy = 0.965054
Epoch 248
Validation binary_cross_entropy = 0.987802
Epoch 249
Loss = 1.1133e-02, PNorm = 32.8259, GNorm = 1.3151, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.981580
Epoch 250
Validation binary_cross_entropy = 0.965449
Epoch 251
Validation binary_cross_entropy = 0.944515
Epoch 252
Validation binary_cross_entropy = 0.925220
Epoch 253
Validation binary_cross_entropy = 0.886467
Epoch 254
Loss = 1.1660e-03, PNorm = 32.8332, GNorm = 0.0057, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.843217
Epoch 255
Validation binary_cross_entropy = 0.820787
Epoch 256
Validation binary_cross_entropy = 0.837530
Epoch 257
Validation binary_cross_entropy = 0.854719
Epoch 258
Validation binary_cross_entropy = 0.875149
Epoch 259
Loss = 4.2660e-03, PNorm = 32.8396, GNorm = 0.4380, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.903276
Epoch 260
Validation binary_cross_entropy = 0.947556
Epoch 261
Validation binary_cross_entropy = 1.021513
Epoch 262
Validation binary_cross_entropy = 1.080098
Epoch 263
Validation binary_cross_entropy = 1.040455
Epoch 264
Loss = 9.9801e-03, PNorm = 32.8482, GNorm = 0.2593, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.949286
Epoch 265
Validation binary_cross_entropy = 0.888319
Epoch 266
Validation binary_cross_entropy = 0.853547
Epoch 267
Validation binary_cross_entropy = 0.844750
Epoch 268
Validation binary_cross_entropy = 0.845204
Epoch 269
Loss = 5.0733e-03, PNorm = 32.8580, GNorm = 0.1533, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.859390
Epoch 270
Validation binary_cross_entropy = 0.872657
Epoch 271
Validation binary_cross_entropy = 0.893185
Epoch 272
Validation binary_cross_entropy = 0.911896
Epoch 273
Validation binary_cross_entropy = 0.930839
Epoch 274
Loss = 2.5945e-03, PNorm = 32.8649, GNorm = 0.2989, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.945507
Epoch 275
Validation binary_cross_entropy = 0.970837
Epoch 276
Validation binary_cross_entropy = 0.974169
Epoch 277
Validation binary_cross_entropy = 0.974291
Epoch 278
Validation binary_cross_entropy = 0.970430
Epoch 279
Loss = 2.5813e-03, PNorm = 32.8700, GNorm = 0.0887, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.970162
Epoch 280
Validation binary_cross_entropy = 0.957578
Epoch 281
Validation binary_cross_entropy = 0.943824
Epoch 282
Validation binary_cross_entropy = 0.938699
Epoch 283
Validation binary_cross_entropy = 0.940687
Epoch 284
Loss = 8.0035e-04, PNorm = 32.8742, GNorm = 0.0492, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.942765
Epoch 285
Validation binary_cross_entropy = 0.944217
Epoch 286
Validation binary_cross_entropy = 0.943368
Epoch 287
Validation binary_cross_entropy = 0.944983
Epoch 288
Validation binary_cross_entropy = 0.950541
Epoch 289
Loss = 7.9239e-03, PNorm = 32.8778, GNorm = 1.2506, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.954222
Epoch 290
Validation binary_cross_entropy = 0.954230
Epoch 291
Validation binary_cross_entropy = 0.955677
Epoch 292
Validation binary_cross_entropy = 0.974226
Epoch 293
Validation binary_cross_entropy = 0.973502
Epoch 294
Loss = 5.8775e-04, PNorm = 32.8811, GNorm = 0.0243, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.975692
Epoch 295
Validation binary_cross_entropy = 0.981124
Epoch 296
Validation binary_cross_entropy = 0.984242
Epoch 297
Validation binary_cross_entropy = 0.977552
Epoch 298
Validation binary_cross_entropy = 0.975719
Epoch 299
Loss = 3.1792e-03, PNorm = 32.8845, GNorm = 0.1242, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.976790
Model 0 best validation binary_cross_entropy = 0.297519 on epoch 4
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.220059
Ensemble test binary_cross_entropy = 0.220059
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.296001
Epoch 1
Validation binary_cross_entropy = 0.345961
Epoch 2
Validation binary_cross_entropy = 1.241372
Epoch 3
Validation binary_cross_entropy = 0.389640
Epoch 4
Loss = 5.8466e-01, PNorm = 31.6261, GNorm = 9.4015, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.275204
Epoch 5
Validation binary_cross_entropy = 0.443292
Epoch 6
Validation binary_cross_entropy = 1.134986
Epoch 7
Validation binary_cross_entropy = 0.435285
Epoch 8
Validation binary_cross_entropy = 0.379354
Epoch 9
Loss = 6.0683e-01, PNorm = 31.6837, GNorm = 13.1645, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.470488
Epoch 10
Validation binary_cross_entropy = 1.049324
Epoch 11
Validation binary_cross_entropy = 0.629556
Epoch 12
Validation binary_cross_entropy = 0.439701
Epoch 13
Validation binary_cross_entropy = 0.401215
Epoch 14
Loss = 2.7372e-01, PNorm = 31.7576, GNorm = 2.6801, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.532951
Epoch 15
Validation binary_cross_entropy = 0.492083
Epoch 16
Validation binary_cross_entropy = 0.343027
Epoch 17
Validation binary_cross_entropy = 0.345388
Epoch 18
Validation binary_cross_entropy = 0.392114
Epoch 19
Loss = 1.6728e-01, PNorm = 31.8186, GNorm = 1.7737, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.556129
Epoch 20
Validation binary_cross_entropy = 0.575056
Epoch 21
Validation binary_cross_entropy = 0.391731
Epoch 22
Validation binary_cross_entropy = 0.397722
Epoch 23
Validation binary_cross_entropy = 0.368097
Epoch 24
Loss = 1.3371e-01, PNorm = 31.8695, GNorm = 2.2245, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.429410
Epoch 25
Validation binary_cross_entropy = 0.466093
Epoch 26
Validation binary_cross_entropy = 0.418695
Epoch 27
Validation binary_cross_entropy = 0.404471
Epoch 28
Validation binary_cross_entropy = 0.401401
Epoch 29
Loss = 1.4962e-01, PNorm = 31.9142, GNorm = 1.6255, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.401282
Epoch 30
Validation binary_cross_entropy = 0.417500
Epoch 31
Validation binary_cross_entropy = 0.443491
Epoch 32
Validation binary_cross_entropy = 0.447510
Epoch 33
Validation binary_cross_entropy = 0.437207
Epoch 34
Loss = 1.2431e-01, PNorm = 31.9517, GNorm = 2.1461, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.424527
Epoch 35
Validation binary_cross_entropy = 0.429361
Epoch 36
Validation binary_cross_entropy = 0.437976
Epoch 37
Validation binary_cross_entropy = 0.453563
Epoch 38
Validation binary_cross_entropy = 0.486372
Epoch 39
Loss = 5.2938e-02, PNorm = 31.9848, GNorm = 2.6345, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.512686
Epoch 40
Validation binary_cross_entropy = 0.506312
Epoch 41
Validation binary_cross_entropy = 0.493476
Epoch 42
Validation binary_cross_entropy = 0.500285
Epoch 43
Validation binary_cross_entropy = 0.496398
Epoch 44
Loss = 9.1493e-02, PNorm = 32.0173, GNorm = 2.3225, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.481011
Epoch 45
Validation binary_cross_entropy = 0.478372
Epoch 46
Validation binary_cross_entropy = 0.473409
Epoch 47
Validation binary_cross_entropy = 0.473225
Epoch 48
Validation binary_cross_entropy = 0.497946
Epoch 49
Loss = 7.8118e-02, PNorm = 32.0517, GNorm = 1.7995, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.510900
Epoch 50
Validation binary_cross_entropy = 0.536387
Epoch 51
Validation binary_cross_entropy = 0.549913
Epoch 52
Validation binary_cross_entropy = 0.552095
Epoch 53
Validation binary_cross_entropy = 0.560138
Epoch 54
Loss = 3.7991e-02, PNorm = 32.0816, GNorm = 2.2408, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.572373
Epoch 55
Validation binary_cross_entropy = 0.582615
Epoch 56
Validation binary_cross_entropy = 0.617116
Epoch 57
Validation binary_cross_entropy = 0.584828
Epoch 58
Validation binary_cross_entropy = 0.561076
Epoch 59
Loss = 6.1115e-02, PNorm = 32.1076, GNorm = 0.8791, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.553391
Epoch 60
Validation binary_cross_entropy = 0.559775
Epoch 61
Validation binary_cross_entropy = 0.562781
Epoch 62
Validation binary_cross_entropy = 0.541150
Epoch 63
Validation binary_cross_entropy = 0.520409
Epoch 64
Loss = 3.2979e-02, PNorm = 32.1361, GNorm = 1.0856, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.507378
Epoch 65
Validation binary_cross_entropy = 0.512693
Epoch 66
Validation binary_cross_entropy = 0.544490
Epoch 67
Validation binary_cross_entropy = 0.535359
Epoch 68
Validation binary_cross_entropy = 0.581289
Epoch 69
Loss = 1.6946e-01, PNorm = 32.1677, GNorm = 4.7008, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.518140
Epoch 70
Validation binary_cross_entropy = 0.617023
Epoch 71
Validation binary_cross_entropy = 0.624985
Epoch 72
Validation binary_cross_entropy = 0.567566
Epoch 73
Validation binary_cross_entropy = 0.617781
Epoch 74
Loss = 4.6238e-02, PNorm = 32.2142, GNorm = 2.3322, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.682290
Epoch 75
Validation binary_cross_entropy = 0.652169
Epoch 76
Validation binary_cross_entropy = 0.699968
Epoch 77
Validation binary_cross_entropy = 0.743452
Epoch 78
Validation binary_cross_entropy = 0.727812
Epoch 79
Loss = 4.0979e-02, PNorm = 32.2610, GNorm = 1.2300, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.734738
Epoch 80
Validation binary_cross_entropy = 0.661599
Epoch 81
Validation binary_cross_entropy = 0.633434
Epoch 82
Validation binary_cross_entropy = 0.589409
Epoch 83
Validation binary_cross_entropy = 0.527349
Epoch 84
Loss = 4.8284e-02, PNorm = 32.3034, GNorm = 2.2497, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.475981
Epoch 85
Validation binary_cross_entropy = 0.473035
Epoch 86
Validation binary_cross_entropy = 0.491335
Epoch 87
Validation binary_cross_entropy = 0.530033
Epoch 88
Validation binary_cross_entropy = 0.576469
Epoch 89
Loss = 8.5484e-02, PNorm = 32.3402, GNorm = 1.6376, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.594416
Epoch 90
Validation binary_cross_entropy = 0.563749
Epoch 91
Validation binary_cross_entropy = 0.556272
Epoch 92
Validation binary_cross_entropy = 0.544402
Epoch 93
Validation binary_cross_entropy = 0.576552
Epoch 94
Loss = 5.9499e-02, PNorm = 32.3726, GNorm = 2.7292, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.591799
Epoch 95
Validation binary_cross_entropy = 0.597086
Epoch 96
Validation binary_cross_entropy = 0.605421
Epoch 97
Validation binary_cross_entropy = 0.611446
Epoch 98
Validation binary_cross_entropy = 0.606889
Epoch 99
Loss = 4.4895e-02, PNorm = 32.4001, GNorm = 2.0120, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.589439
Epoch 100
Validation binary_cross_entropy = 0.579259
Epoch 101
Validation binary_cross_entropy = 0.589717
Epoch 102
Validation binary_cross_entropy = 0.593109
Epoch 103
Validation binary_cross_entropy = 0.567211
Epoch 104
Loss = 1.2141e-01, PNorm = 32.4225, GNorm = 2.8996, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.606116
Epoch 105
Validation binary_cross_entropy = 0.650715
Epoch 106
Validation binary_cross_entropy = 0.635796
Epoch 107
Validation binary_cross_entropy = 0.605381
Epoch 108
Validation binary_cross_entropy = 0.586579
Epoch 109
Loss = 1.7936e-02, PNorm = 32.4444, GNorm = 0.1582, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.590312
Epoch 110
Validation binary_cross_entropy = 0.592947
Epoch 111
Validation binary_cross_entropy = 0.608198
Epoch 112
Validation binary_cross_entropy = 0.635826
Epoch 113
Validation binary_cross_entropy = 0.670512
Epoch 114
Loss = 1.7461e-02, PNorm = 32.4623, GNorm = 0.7999, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.696434
Epoch 115
Validation binary_cross_entropy = 0.669045
Epoch 116
Validation binary_cross_entropy = 0.663731
Epoch 117
Validation binary_cross_entropy = 0.669077
Epoch 118
Validation binary_cross_entropy = 0.663756
Epoch 119
Loss = 6.0568e-03, PNorm = 32.4812, GNorm = 0.5134, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.657496
Epoch 120
Validation binary_cross_entropy = 0.669640
Epoch 121
Validation binary_cross_entropy = 0.683905
Epoch 122
Validation binary_cross_entropy = 0.727478
Epoch 123
Validation binary_cross_entropy = 0.765200
Epoch 124
Loss = 6.3400e-02, PNorm = 32.4971, GNorm = 1.8159, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.722212
Epoch 125
Validation binary_cross_entropy = 0.660830
Epoch 126
Validation binary_cross_entropy = 0.632881
Epoch 127
Validation binary_cross_entropy = 0.643840
Epoch 128
Validation binary_cross_entropy = 0.693112
Epoch 129
Loss = 3.6763e-02, PNorm = 32.5144, GNorm = 0.6595, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.760944
Epoch 130
Validation binary_cross_entropy = 0.822550
Epoch 131
Validation binary_cross_entropy = 0.819671
Epoch 132
Validation binary_cross_entropy = 0.764840
Epoch 133
Validation binary_cross_entropy = 0.702347
Epoch 134
Loss = 4.4211e-03, PNorm = 32.5314, GNorm = 0.2279, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.662499
Epoch 135
Validation binary_cross_entropy = 0.644904
Epoch 136
Validation binary_cross_entropy = 0.650264
Epoch 137
Validation binary_cross_entropy = 0.662289
Epoch 138
Validation binary_cross_entropy = 0.680490
Epoch 139
Loss = 6.3617e-03, PNorm = 32.5472, GNorm = 0.3213, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.698689
Epoch 140
Validation binary_cross_entropy = 0.713297
Epoch 141
Validation binary_cross_entropy = 0.699845
Epoch 142
Validation binary_cross_entropy = 0.676553
Epoch 143
Validation binary_cross_entropy = 0.657892
Epoch 144
Loss = 6.2565e-03, PNorm = 32.5611, GNorm = 0.4668, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.648938
Epoch 145
Validation binary_cross_entropy = 0.662178
Epoch 146
Validation binary_cross_entropy = 0.667845
Epoch 147
Validation binary_cross_entropy = 0.681273
Epoch 148
Validation binary_cross_entropy = 0.695218
Epoch 149
Loss = 2.6495e-02, PNorm = 32.5742, GNorm = 3.1215, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.715336
Epoch 150
Validation binary_cross_entropy = 0.734773
Epoch 151
Validation binary_cross_entropy = 0.741216
Epoch 152
Validation binary_cross_entropy = 0.720859
Epoch 153
Validation binary_cross_entropy = 0.717120
Epoch 154
Loss = 4.4970e-02, PNorm = 32.5882, GNorm = 3.0372, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.731731
Epoch 155
Validation binary_cross_entropy = 0.711831
Epoch 156
Validation binary_cross_entropy = 0.690452
Epoch 157
Validation binary_cross_entropy = 0.673538
Epoch 158
Validation binary_cross_entropy = 0.653215
Epoch 159
Loss = 2.2714e-02, PNorm = 32.6022, GNorm = 1.6621, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.646127
Epoch 160
Validation binary_cross_entropy = 0.632827
Epoch 161
Validation binary_cross_entropy = 0.630233
Epoch 162
Validation binary_cross_entropy = 0.639146
Epoch 163
Validation binary_cross_entropy = 0.639690
Epoch 164
Loss = 2.7907e-02, PNorm = 32.6196, GNorm = 0.5476, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.640982
Epoch 165
Validation binary_cross_entropy = 0.645848
Epoch 166
Validation binary_cross_entropy = 0.663116
Epoch 167
Validation binary_cross_entropy = 0.680489
Epoch 168
Validation binary_cross_entropy = 0.707474
Epoch 169
Loss = 1.6897e-02, PNorm = 32.6350, GNorm = 2.1902, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.743870
Epoch 170
Validation binary_cross_entropy = 0.800873
Epoch 171
Validation binary_cross_entropy = 0.820798
Epoch 172
Validation binary_cross_entropy = 0.809654
Epoch 173
Validation binary_cross_entropy = 0.790310
Epoch 174
Loss = 3.2917e-03, PNorm = 32.6490, GNorm = 0.0848, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.777144
Epoch 175
Validation binary_cross_entropy = 0.780124
Epoch 176
Validation binary_cross_entropy = 0.802635
Epoch 177
Validation binary_cross_entropy = 0.813535
Epoch 178
Validation binary_cross_entropy = 0.816575
Epoch 179
Loss = 2.4091e-02, PNorm = 32.6679, GNorm = 2.0808, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.790783
Epoch 180
Validation binary_cross_entropy = 0.726873
Epoch 181
Validation binary_cross_entropy = 0.682552
Epoch 182
Validation binary_cross_entropy = 0.675474
Epoch 183
Validation binary_cross_entropy = 0.688561
Epoch 184
Loss = 1.4719e-02, PNorm = 32.6932, GNorm = 1.6137, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.713089
Epoch 185
Validation binary_cross_entropy = 0.736682
Epoch 186
Validation binary_cross_entropy = 0.759817
Epoch 187
Validation binary_cross_entropy = 0.722031
Epoch 188
Validation binary_cross_entropy = 0.697737
Epoch 189
Loss = 4.4539e-03, PNorm = 32.7132, GNorm = 0.2074, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.688447
Epoch 190
Validation binary_cross_entropy = 0.693072
Epoch 191
Validation binary_cross_entropy = 0.726625
Epoch 192
Validation binary_cross_entropy = 0.782922
Epoch 193
Validation binary_cross_entropy = 0.820532
Epoch 194
Loss = 8.3840e-03, PNorm = 32.7296, GNorm = 0.5288, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.831517
Epoch 195
Validation binary_cross_entropy = 0.751348
Epoch 196
Validation binary_cross_entropy = 0.701223
Epoch 197
Validation binary_cross_entropy = 0.673960
Epoch 198
Validation binary_cross_entropy = 0.665315
Epoch 199
Loss = 1.2028e-02, PNorm = 32.7475, GNorm = 1.5704, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.673461
Epoch 200
Validation binary_cross_entropy = 0.694094
Epoch 201
Validation binary_cross_entropy = 0.718860
Epoch 202
Validation binary_cross_entropy = 0.746120
Epoch 203
Validation binary_cross_entropy = 0.759960
Epoch 204
Loss = 6.3768e-03, PNorm = 32.7645, GNorm = 0.5436, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.770198
Epoch 205
Validation binary_cross_entropy = 0.795308
Epoch 206
Validation binary_cross_entropy = 0.826000
Epoch 207
Validation binary_cross_entropy = 0.807497
Epoch 208
Validation binary_cross_entropy = 0.753134
Epoch 209
Loss = 5.3089e-02, PNorm = 32.7817, GNorm = 3.1210, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.750531
Epoch 210
Validation binary_cross_entropy = 0.759975
Epoch 211
Validation binary_cross_entropy = 0.769154
Epoch 212
Validation binary_cross_entropy = 0.756317
Epoch 213
Validation binary_cross_entropy = 0.754527
Epoch 214
Loss = 9.4462e-03, PNorm = 32.8007, GNorm = 0.4385, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.767554
Epoch 215
Validation binary_cross_entropy = 0.766317
Epoch 216
Validation binary_cross_entropy = 0.725755
Epoch 217
Validation binary_cross_entropy = 0.693684
Epoch 218
Validation binary_cross_entropy = 0.672490
Epoch 219
Loss = 7.9887e-03, PNorm = 32.8171, GNorm = 0.2960, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.672596
Epoch 220
Validation binary_cross_entropy = 0.688812
Epoch 221
Validation binary_cross_entropy = 0.706212
Epoch 222
Validation binary_cross_entropy = 0.717242
Epoch 223
Validation binary_cross_entropy = 0.732048
Epoch 224
Loss = 7.0466e-03, PNorm = 32.8290, GNorm = 0.0598, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.742840
Epoch 225
Validation binary_cross_entropy = 0.769343
Epoch 226
Validation binary_cross_entropy = 0.790258
Epoch 227
Validation binary_cross_entropy = 0.799895
Epoch 228
Validation binary_cross_entropy = 0.793403
Epoch 229
Loss = 1.5213e-02, PNorm = 32.8379, GNorm = 0.8630, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.758815
Epoch 230
Validation binary_cross_entropy = 0.716960
Epoch 231
Validation binary_cross_entropy = 0.693264
Epoch 232
Validation binary_cross_entropy = 0.688382
Epoch 233
Validation binary_cross_entropy = 0.692769
Epoch 234
Loss = 8.5778e-03, PNorm = 32.8459, GNorm = 0.1629, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.715010
Epoch 235
Validation binary_cross_entropy = 0.735219
Epoch 236
Validation binary_cross_entropy = 0.753593
Epoch 237
Validation binary_cross_entropy = 0.764512
Epoch 238
Validation binary_cross_entropy = 0.744158
Epoch 239
Loss = 5.8459e-03, PNorm = 32.8537, GNorm = 0.1599, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.737216
Epoch 240
Validation binary_cross_entropy = 0.740342
Epoch 241
Validation binary_cross_entropy = 0.747276
Epoch 242
Validation binary_cross_entropy = 0.759040
Epoch 243
Validation binary_cross_entropy = 0.774912
Epoch 244
Loss = 2.7364e-02, PNorm = 32.8598, GNorm = 1.4080, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.810812
Epoch 245
Validation binary_cross_entropy = 0.857402
Epoch 246
Validation binary_cross_entropy = 0.877862
Epoch 247
Validation binary_cross_entropy = 0.839748
Epoch 248
Validation binary_cross_entropy = 0.803087
Epoch 249
Loss = 2.4164e-03, PNorm = 32.8690, GNorm = 0.0800, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.774888
Epoch 250
Validation binary_cross_entropy = 0.758834
Epoch 251
Validation binary_cross_entropy = 0.759273
Epoch 252
Validation binary_cross_entropy = 0.771892
Epoch 253
Validation binary_cross_entropy = 0.789141
Epoch 254
Loss = 1.1367e-02, PNorm = 32.8782, GNorm = 0.4643, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.798065
Epoch 255
Validation binary_cross_entropy = 0.790512
Epoch 256
Validation binary_cross_entropy = 0.780001
Epoch 257
Validation binary_cross_entropy = 0.777308
Epoch 258
Validation binary_cross_entropy = 0.775699
Epoch 259
Loss = 1.4734e-03, PNorm = 32.8864, GNorm = 0.0769, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.779134
Epoch 260
Validation binary_cross_entropy = 0.784928
Epoch 261
Validation binary_cross_entropy = 0.801482
Epoch 262
Validation binary_cross_entropy = 0.815261
Epoch 263
Validation binary_cross_entropy = 0.829895
Epoch 264
Loss = 4.2910e-03, PNorm = 32.8924, GNorm = 0.0966, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.844669
Epoch 265
Validation binary_cross_entropy = 0.852749
Epoch 266
Validation binary_cross_entropy = 0.842804
Epoch 267
Validation binary_cross_entropy = 0.839488
Epoch 268
Validation binary_cross_entropy = 0.848275
Epoch 269
Loss = 1.8672e-03, PNorm = 32.8977, GNorm = 0.1012, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.858055
Epoch 270
Validation binary_cross_entropy = 0.868839
Epoch 271
Validation binary_cross_entropy = 0.874040
Epoch 272
Validation binary_cross_entropy = 0.865258
Epoch 273
Validation binary_cross_entropy = 0.858174
Epoch 274
Loss = 7.8933e-04, PNorm = 32.9038, GNorm = 0.0278, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.854629
Epoch 275
Validation binary_cross_entropy = 0.848210
Epoch 276
Validation binary_cross_entropy = 0.848002
Epoch 277
Validation binary_cross_entropy = 0.853436
Epoch 278
Validation binary_cross_entropy = 0.853962
Epoch 279
Loss = 2.0870e-02, PNorm = 32.9091, GNorm = 0.4335, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.824083
Epoch 280
Validation binary_cross_entropy = 0.809355
Epoch 281
Validation binary_cross_entropy = 0.808305
Epoch 282
Validation binary_cross_entropy = 0.811615
Epoch 283
Validation binary_cross_entropy = 0.856313
Epoch 284
Loss = 3.8715e-03, PNorm = 32.9182, GNorm = 0.5787, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.917080
Epoch 285
Validation binary_cross_entropy = 0.966051
Epoch 286
Validation binary_cross_entropy = 0.972825
Epoch 287
Validation binary_cross_entropy = 0.892946
Epoch 288
Validation binary_cross_entropy = 0.838086
Epoch 289
Loss = 1.7831e-02, PNorm = 32.9304, GNorm = 1.9737, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.828386
Epoch 290
Validation binary_cross_entropy = 0.840217
Epoch 291
Validation binary_cross_entropy = 0.838006
Epoch 292
Validation binary_cross_entropy = 0.834847
Epoch 293
Validation binary_cross_entropy = 0.814561
Epoch 294
Loss = 5.8647e-03, PNorm = 32.9457, GNorm = 0.0057, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.784007
Epoch 295
Validation binary_cross_entropy = 0.762924
Epoch 296
Validation binary_cross_entropy = 0.749893
Epoch 297
Validation binary_cross_entropy = 0.760219
Epoch 298
Validation binary_cross_entropy = 0.793022
Epoch 299
Loss = 4.6095e-03, PNorm = 32.9586, GNorm = 0.1687, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.827579
Model 0 best validation binary_cross_entropy = 0.275204 on epoch 4
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.243466
Ensemble test binary_cross_entropy = 0.243466
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.328106
Epoch 1
Validation binary_cross_entropy = 0.338134
Epoch 2
Validation binary_cross_entropy = 0.743898
Epoch 3
Validation binary_cross_entropy = 0.359818
Epoch 4
Loss = 6.4243e-01, PNorm = 31.6234, GNorm = 11.4548, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.355859
Epoch 5
Validation binary_cross_entropy = 0.606574
Epoch 6
Validation binary_cross_entropy = 1.142658
Epoch 7
Validation binary_cross_entropy = 0.460145
Epoch 8
Validation binary_cross_entropy = 0.340077
Epoch 9
Loss = 4.9847e-01, PNorm = 31.6835, GNorm = 7.1358, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.592352
Epoch 10
Validation binary_cross_entropy = 0.938807
Epoch 11
Validation binary_cross_entropy = 0.319880
Epoch 12
Validation binary_cross_entropy = 0.327051
Epoch 13
Validation binary_cross_entropy = 0.397738
Epoch 14
Loss = 2.5737e-01, PNorm = 31.7629, GNorm = 4.4333, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.718250
Epoch 15
Validation binary_cross_entropy = 0.493533
Epoch 16
Validation binary_cross_entropy = 0.374700
Epoch 17
Validation binary_cross_entropy = 0.386146
Epoch 18
Validation binary_cross_entropy = 0.402152
Epoch 19
Loss = 1.9597e-01, PNorm = 31.8310, GNorm = 2.8788, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.442794
Epoch 20
Validation binary_cross_entropy = 0.448197
Epoch 21
Validation binary_cross_entropy = 0.410522
Epoch 22
Validation binary_cross_entropy = 0.414523
Epoch 23
Validation binary_cross_entropy = 0.425357
Epoch 24
Loss = 2.0665e-01, PNorm = 31.8842, GNorm = 1.8316, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.445849
Epoch 25
Validation binary_cross_entropy = 0.469397
Epoch 26
Validation binary_cross_entropy = 0.515471
Epoch 27
Validation binary_cross_entropy = 0.509388
Epoch 28
Validation binary_cross_entropy = 0.471526
Epoch 29
Loss = 1.6697e-01, PNorm = 31.9285, GNorm = 2.3435, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.457192
Epoch 30
Validation binary_cross_entropy = 0.447178
Epoch 31
Validation binary_cross_entropy = 0.452937
Epoch 32
Validation binary_cross_entropy = 0.451535
Epoch 33
Validation binary_cross_entropy = 0.457371
Epoch 34
Loss = 1.2206e-01, PNorm = 31.9687, GNorm = 2.4815, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.463851
Epoch 35
Validation binary_cross_entropy = 0.478091
Epoch 36
Validation binary_cross_entropy = 0.512647
Epoch 37
Validation binary_cross_entropy = 0.501659
Epoch 38
Validation binary_cross_entropy = 0.444786
Epoch 39
Loss = 9.8139e-02, PNorm = 32.0065, GNorm = 1.2018, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.489444
Epoch 40
Validation binary_cross_entropy = 0.442342
Epoch 41
Validation binary_cross_entropy = 0.452108
Epoch 42
Validation binary_cross_entropy = 0.481143
Epoch 43
Validation binary_cross_entropy = 0.445538
Epoch 44
Loss = 7.1353e-02, PNorm = 32.0505, GNorm = 1.8090, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.519267
Epoch 45
Validation binary_cross_entropy = 0.539345
Epoch 46
Validation binary_cross_entropy = 0.515595
Epoch 47
Validation binary_cross_entropy = 0.552566
Epoch 48
Validation binary_cross_entropy = 0.551803
Epoch 49
Loss = 1.3674e-01, PNorm = 32.0926, GNorm = 3.2109, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.481948
Epoch 50
Validation binary_cross_entropy = 0.490179
Epoch 51
Validation binary_cross_entropy = 0.497686
Epoch 52
Validation binary_cross_entropy = 0.498434
Epoch 53
Validation binary_cross_entropy = 0.572898
Epoch 54
Loss = 9.8426e-02, PNorm = 32.1341, GNorm = 2.3688, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.569985
Epoch 55
Validation binary_cross_entropy = 0.558474
Epoch 56
Validation binary_cross_entropy = 0.556188
Epoch 57
Validation binary_cross_entropy = 0.543421
Epoch 58
Validation binary_cross_entropy = 0.553423
Epoch 59
Loss = 8.7595e-02, PNorm = 32.1710, GNorm = 3.5130, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.556308
Epoch 60
Validation binary_cross_entropy = 0.536916
Epoch 61
Validation binary_cross_entropy = 0.521320
Epoch 62
Validation binary_cross_entropy = 0.519670
Epoch 63
Validation binary_cross_entropy = 0.536003
Epoch 64
Loss = 2.7184e-02, PNorm = 32.2048, GNorm = 0.5822, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.573797
Epoch 65
Validation binary_cross_entropy = 0.565965
Epoch 66
Validation binary_cross_entropy = 0.550585
Epoch 67
Validation binary_cross_entropy = 0.555686
Epoch 68
Validation binary_cross_entropy = 0.564395
Epoch 69
Loss = 5.1696e-02, PNorm = 32.2359, GNorm = 1.3194, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.567612
Epoch 70
Validation binary_cross_entropy = 0.590055
Epoch 71
Validation binary_cross_entropy = 0.628175
Epoch 72
Validation binary_cross_entropy = 0.639356
Epoch 73
Validation binary_cross_entropy = 0.619508
Epoch 74
Loss = 4.6587e-02, PNorm = 32.2632, GNorm = 2.1122, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.600130
Epoch 75
Validation binary_cross_entropy = 0.581247
Epoch 76
Validation binary_cross_entropy = 0.578137
Epoch 77
Validation binary_cross_entropy = 0.580793
Epoch 78
Validation binary_cross_entropy = 0.580725
Epoch 79
Loss = 2.5272e-02, PNorm = 32.2862, GNorm = 1.2706, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.590859
Epoch 80
Validation binary_cross_entropy = 0.637560
Epoch 81
Validation binary_cross_entropy = 0.694436
Epoch 82
Validation binary_cross_entropy = 0.654985
Epoch 83
Validation binary_cross_entropy = 0.607161
Epoch 84
Loss = 8.5966e-02, PNorm = 32.3079, GNorm = 4.4951, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.597774
Epoch 85
Validation binary_cross_entropy = 0.608547
Epoch 86
Validation binary_cross_entropy = 0.628374
Epoch 87
Validation binary_cross_entropy = 0.629583
Epoch 88
Validation binary_cross_entropy = 0.622824
Epoch 89
Loss = 2.4372e-02, PNorm = 32.3270, GNorm = 1.2394, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.643567
Epoch 90
Validation binary_cross_entropy = 0.644865
Epoch 91
Validation binary_cross_entropy = 0.596155
Epoch 92
Validation binary_cross_entropy = 0.576310
Epoch 93
Validation binary_cross_entropy = 0.562120
Epoch 94
Loss = 1.1918e-02, PNorm = 32.3503, GNorm = 0.8265, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.555541
Epoch 95
Validation binary_cross_entropy = 0.554798
Epoch 96
Validation binary_cross_entropy = 0.553610
Epoch 97
Validation binary_cross_entropy = 0.601205
Epoch 98
Validation binary_cross_entropy = 0.654779
Epoch 99
Loss = 5.7249e-02, PNorm = 32.3724, GNorm = 0.2373, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.604559
Epoch 100
Validation binary_cross_entropy = 0.609699
Epoch 101
Validation binary_cross_entropy = 0.622751
Epoch 102
Validation binary_cross_entropy = 0.652096
Epoch 103
Validation binary_cross_entropy = 0.668832
Epoch 104
Loss = 8.6022e-02, PNorm = 32.3965, GNorm = 3.3654, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.655188
Epoch 105
Validation binary_cross_entropy = 0.653461
Epoch 106
Validation binary_cross_entropy = 0.667127
Epoch 107
Validation binary_cross_entropy = 0.631345
Epoch 108
Validation binary_cross_entropy = 0.611131
Epoch 109
Loss = 2.6428e-02, PNorm = 32.4227, GNorm = 1.0781, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.607620
Epoch 110
Validation binary_cross_entropy = 0.612574
Epoch 111
Validation binary_cross_entropy = 0.615122
Epoch 112
Validation binary_cross_entropy = 0.620702
Epoch 113
Validation binary_cross_entropy = 0.621920
Epoch 114
Loss = 2.1003e-02, PNorm = 32.4467, GNorm = 0.7684, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.610083
Epoch 115
Validation binary_cross_entropy = 0.602011
Epoch 116
Validation binary_cross_entropy = 0.651308
Epoch 117
Validation binary_cross_entropy = 0.747339
Epoch 118
Validation binary_cross_entropy = 0.677951
Epoch 119
Loss = 8.6131e-03, PNorm = 32.4697, GNorm = 0.4219, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.622492
Epoch 120
Validation binary_cross_entropy = 0.639091
Epoch 121
Validation binary_cross_entropy = 0.658463
Epoch 122
Validation binary_cross_entropy = 0.689584
Epoch 123
Validation binary_cross_entropy = 0.719769
Epoch 124
Loss = 2.8543e-02, PNorm = 32.4957, GNorm = 0.2805, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.698580
Epoch 125
Validation binary_cross_entropy = 0.669792
Epoch 126
Validation binary_cross_entropy = 0.647875
Epoch 127
Validation binary_cross_entropy = 0.636627
Epoch 128
Validation binary_cross_entropy = 0.659038
Epoch 129
Loss = 2.3383e-02, PNorm = 32.5209, GNorm = 0.2569, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.664641
Epoch 130
Validation binary_cross_entropy = 0.733236
Epoch 131
Validation binary_cross_entropy = 0.736721
Epoch 132
Validation binary_cross_entropy = 0.683803
Epoch 133
Validation binary_cross_entropy = 0.680714
Epoch 134
Loss = 8.9612e-02, PNorm = 32.5479, GNorm = 2.4505, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.703280
Epoch 135
Validation binary_cross_entropy = 0.693888
Epoch 136
Validation binary_cross_entropy = 0.744710
Epoch 137
Validation binary_cross_entropy = 0.804129
Epoch 138
Validation binary_cross_entropy = 0.753535
Epoch 139
Loss = 5.0369e-02, PNorm = 32.5864, GNorm = 2.7310, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.716586
Epoch 140
Validation binary_cross_entropy = 0.758707
Epoch 141
Validation binary_cross_entropy = 0.801597
Epoch 142
Validation binary_cross_entropy = 0.802071
Epoch 143
Validation binary_cross_entropy = 0.810433
Epoch 144
Loss = 1.9997e-02, PNorm = 32.6212, GNorm = 0.4036, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.800869
Epoch 145
Validation binary_cross_entropy = 0.780995
Epoch 146
Validation binary_cross_entropy = 0.772182
Epoch 147
Validation binary_cross_entropy = 0.752978
Epoch 148
Validation binary_cross_entropy = 0.767334
Epoch 149
Loss = 4.0574e-02, PNorm = 32.6531, GNorm = 0.3175, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.816376
Epoch 150
Validation binary_cross_entropy = 0.851310
Epoch 151
Validation binary_cross_entropy = 0.827612
Epoch 152
Validation binary_cross_entropy = 0.810784
Epoch 153
Validation binary_cross_entropy = 0.809727
Epoch 154
Loss = 3.8018e-02, PNorm = 32.6879, GNorm = 1.9428, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.818855
Epoch 155
Validation binary_cross_entropy = 0.811336
Epoch 156
Validation binary_cross_entropy = 0.802185
Epoch 157
Validation binary_cross_entropy = 0.812644
Epoch 158
Validation binary_cross_entropy = 0.805797
Epoch 159
Loss = 4.8208e-02, PNorm = 32.7254, GNorm = 0.8104, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.801559
Epoch 160
Validation binary_cross_entropy = 0.787703
Epoch 161
Validation binary_cross_entropy = 0.782358
Epoch 162
Validation binary_cross_entropy = 0.766059
Epoch 163
Validation binary_cross_entropy = 0.830292
Epoch 164
Loss = 5.6447e-02, PNorm = 32.7596, GNorm = 3.8408, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.881305
Epoch 165
Validation binary_cross_entropy = 0.781859
Epoch 166
Validation binary_cross_entropy = 0.714449
Epoch 167
Validation binary_cross_entropy = 0.702160
Epoch 168
Validation binary_cross_entropy = 0.709070
Epoch 169
Loss = 8.2303e-02, PNorm = 32.7897, GNorm = 3.4313, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.716819
Epoch 170
Validation binary_cross_entropy = 0.719033
Epoch 171
Validation binary_cross_entropy = 0.712676
Epoch 172
Validation binary_cross_entropy = 0.709709
Epoch 173
Validation binary_cross_entropy = 0.711119
Epoch 174
Loss = 2.3837e-02, PNorm = 32.8135, GNorm = 0.9938, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.719210
Epoch 175
Validation binary_cross_entropy = 0.718961
Epoch 176
Validation binary_cross_entropy = 0.701085
Epoch 177
Validation binary_cross_entropy = 0.689401
Epoch 178
Validation binary_cross_entropy = 0.682869
Epoch 179
Loss = 2.4666e-02, PNorm = 32.8297, GNorm = 0.2214, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.696705
Epoch 180
Validation binary_cross_entropy = 0.722251
Epoch 181
Validation binary_cross_entropy = 0.741839
Epoch 182
Validation binary_cross_entropy = 0.757886
Epoch 183
Validation binary_cross_entropy = 0.774504
Epoch 184
Loss = 8.4955e-03, PNorm = 32.8444, GNorm = 0.9837, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.797818
Epoch 185
Validation binary_cross_entropy = 0.790770
Epoch 186
Validation binary_cross_entropy = 0.777972
Epoch 187
Validation binary_cross_entropy = 0.772497
Epoch 188
Validation binary_cross_entropy = 0.772008
Epoch 189
Loss = 5.3380e-03, PNorm = 32.8566, GNorm = 0.0819, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.772424
Epoch 190
Validation binary_cross_entropy = 0.774206
Epoch 191
Validation binary_cross_entropy = 0.754644
Epoch 192
Validation binary_cross_entropy = 0.759432
Epoch 193
Validation binary_cross_entropy = 0.790440
Epoch 194
Loss = 4.6204e-02, PNorm = 32.8728, GNorm = 1.7393, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.813893
Epoch 195
Validation binary_cross_entropy = 0.827606
Epoch 196
Validation binary_cross_entropy = 0.833251
Epoch 197
Validation binary_cross_entropy = 0.828140
Epoch 198
Validation binary_cross_entropy = 0.814043
Epoch 199
Loss = 3.5243e-03, PNorm = 32.8886, GNorm = 0.1296, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.796588
Epoch 200
Validation binary_cross_entropy = 0.782155
Epoch 201
Validation binary_cross_entropy = 0.773056
Epoch 202
Validation binary_cross_entropy = 0.774541
Epoch 203
Validation binary_cross_entropy = 0.776971
Epoch 204
Loss = 2.6803e-03, PNorm = 32.8987, GNorm = 0.0436, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.780739
Epoch 205
Validation binary_cross_entropy = 0.791116
Epoch 206
Validation binary_cross_entropy = 0.799594
Epoch 207
Validation binary_cross_entropy = 0.802708
Epoch 208
Validation binary_cross_entropy = 0.808047
Epoch 209
Loss = 1.5676e-03, PNorm = 32.9069, GNorm = 0.0335, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.815009
Epoch 210
Validation binary_cross_entropy = 0.821877
Epoch 211
Validation binary_cross_entropy = 0.832542
Epoch 212
Validation binary_cross_entropy = 0.844757
Epoch 213
Validation binary_cross_entropy = 0.858823
Epoch 214
Loss = 1.0734e-03, PNorm = 32.9135, GNorm = 0.0376, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.870750
Epoch 215
Validation binary_cross_entropy = 0.884187
Epoch 216
Validation binary_cross_entropy = 0.897019
Epoch 217
Validation binary_cross_entropy = 0.897427
Epoch 218
Validation binary_cross_entropy = 0.888964
Epoch 219
Loss = 4.6535e-02, PNorm = 32.9180, GNorm = 2.0869, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.891735
Epoch 220
Validation binary_cross_entropy = 0.912586
Epoch 221
Validation binary_cross_entropy = 0.927122
Epoch 222
Validation binary_cross_entropy = 0.915974
Epoch 223
Validation binary_cross_entropy = 0.889464
Epoch 224
Loss = 7.2700e-03, PNorm = 32.9231, GNorm = 1.0331, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.866953
Epoch 225
Validation binary_cross_entropy = 0.862880
Epoch 226
Validation binary_cross_entropy = 0.866025
Epoch 227
Validation binary_cross_entropy = 0.869556
Epoch 228
Validation binary_cross_entropy = 0.878714
Epoch 229
Loss = 2.0391e-03, PNorm = 32.9297, GNorm = 0.0396, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.889686
Epoch 230
Validation binary_cross_entropy = 0.897507
Epoch 231
Validation binary_cross_entropy = 0.917478
Epoch 232
Validation binary_cross_entropy = 0.947896
Epoch 233
Validation binary_cross_entropy = 0.973494
Epoch 234
Loss = 4.2617e-03, PNorm = 32.9368, GNorm = 0.4388, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.983841
Epoch 235
Validation binary_cross_entropy = 0.955640
Epoch 236
Validation binary_cross_entropy = 0.915760
Epoch 237
Validation binary_cross_entropy = 0.886954
Epoch 238
Validation binary_cross_entropy = 0.874861
Epoch 239
Loss = 3.0147e-03, PNorm = 32.9436, GNorm = 0.3910, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.874450
Epoch 240
Validation binary_cross_entropy = 0.876537
Epoch 241
Validation binary_cross_entropy = 0.892139
Epoch 242
Validation binary_cross_entropy = 0.912905
Epoch 243
Validation binary_cross_entropy = 0.931799
Epoch 244
Loss = 5.8892e-03, PNorm = 32.9512, GNorm = 0.1482, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.936398
Epoch 245
Validation binary_cross_entropy = 0.916415
Epoch 246
Validation binary_cross_entropy = 0.899758
Epoch 247
Validation binary_cross_entropy = 0.887762
Epoch 248
Validation binary_cross_entropy = 0.881256
Epoch 249
Loss = 2.3657e-03, PNorm = 32.9574, GNorm = 0.0183, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.874849
Epoch 250
Validation binary_cross_entropy = 0.875416
Epoch 251
Validation binary_cross_entropy = 0.896103
Epoch 252
Validation binary_cross_entropy = 0.916703
Epoch 253
Validation binary_cross_entropy = 0.932066
Epoch 254
Loss = 5.4812e-02, PNorm = 32.9619, GNorm = 3.0918, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.919196
Epoch 255
Validation binary_cross_entropy = 0.891212
Epoch 256
Validation binary_cross_entropy = 0.868397
Epoch 257
Validation binary_cross_entropy = 0.853614
Epoch 258
Validation binary_cross_entropy = 0.847944
Epoch 259
Loss = 2.3479e-03, PNorm = 32.9666, GNorm = 0.0301, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.851093
Epoch 260
Validation binary_cross_entropy = 0.855462
Epoch 261
Validation binary_cross_entropy = 0.862168
Epoch 262
Validation binary_cross_entropy = 0.870464
Epoch 263
Validation binary_cross_entropy = 0.879228
Epoch 264
Loss = 1.3687e-03, PNorm = 32.9703, GNorm = 0.0804, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.885542
Epoch 265
Validation binary_cross_entropy = 0.891865
Epoch 266
Validation binary_cross_entropy = 0.897660
Epoch 267
Validation binary_cross_entropy = 0.909605
Epoch 268
Validation binary_cross_entropy = 0.925988
Epoch 269
Loss = 6.1024e-04, PNorm = 32.9729, GNorm = 0.0225, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.939948
Epoch 270
Validation binary_cross_entropy = 0.945037
Epoch 271
Validation binary_cross_entropy = 0.931440
Epoch 272
Validation binary_cross_entropy = 0.919069
Epoch 273
Validation binary_cross_entropy = 0.910862
Epoch 274
Loss = 8.8224e-04, PNorm = 32.9765, GNorm = 0.0320, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.905781
Epoch 275
Validation binary_cross_entropy = 0.903200
Epoch 276
Validation binary_cross_entropy = 0.909462
Epoch 277
Validation binary_cross_entropy = 0.923594
Epoch 278
Validation binary_cross_entropy = 0.930516
Epoch 279
Loss = 3.0485e-02, PNorm = 32.9801, GNorm = 0.0663, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.902560
Epoch 280
Validation binary_cross_entropy = 0.878506
Epoch 281
Validation binary_cross_entropy = 0.861408
Epoch 282
Validation binary_cross_entropy = 0.849831
Epoch 283
Validation binary_cross_entropy = 0.846109
Epoch 284
Loss = 9.1361e-03, PNorm = 32.9858, GNorm = 1.1372, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.852256
Epoch 285
Validation binary_cross_entropy = 0.861171
Epoch 286
Validation binary_cross_entropy = 0.868629
Epoch 287
Validation binary_cross_entropy = 0.883919
Epoch 288
Validation binary_cross_entropy = 0.914742
Epoch 289
Loss = 2.9211e-03, PNorm = 32.9900, GNorm = 0.3278, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.941663
Epoch 290
Validation binary_cross_entropy = 0.959164
Epoch 291
Validation binary_cross_entropy = 0.921444
Epoch 292
Validation binary_cross_entropy = 0.897571
Epoch 293
Validation binary_cross_entropy = 0.880521
Epoch 294
Loss = 1.2374e-03, PNorm = 32.9946, GNorm = 0.0532, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.867519
Epoch 295
Validation binary_cross_entropy = 0.861709
Epoch 296
Validation binary_cross_entropy = 0.865588
Epoch 297
Validation binary_cross_entropy = 0.875515
Epoch 298
Validation binary_cross_entropy = 0.885354
Epoch 299
Loss = 3.3788e-03, PNorm = 32.9984, GNorm = 0.0226, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.892896
Model 0 best validation binary_cross_entropy = 0.319880 on epoch 11
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.243307
Ensemble test binary_cross_entropy = 0.243307
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.308618
Epoch 1
Validation binary_cross_entropy = 0.303287
Epoch 2
Validation binary_cross_entropy = 1.072898
Epoch 3
Validation binary_cross_entropy = 0.541213
Epoch 4
Loss = 5.8069e-01, PNorm = 31.6233, GNorm = 7.1594, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.305258
Epoch 5
Validation binary_cross_entropy = 0.334767
Epoch 6
Validation binary_cross_entropy = 0.669325
Epoch 7
Validation binary_cross_entropy = 0.346773
Epoch 8
Validation binary_cross_entropy = 0.306979
Epoch 9
Loss = 4.3790e-01, PNorm = 31.6796, GNorm = 3.9718, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.406107
Epoch 10
Validation binary_cross_entropy = 0.412300
Epoch 11
Validation binary_cross_entropy = 0.420779
Epoch 12
Validation binary_cross_entropy = 0.460401
Epoch 13
Validation binary_cross_entropy = 0.436728
Epoch 14
Loss = 3.7298e-01, PNorm = 31.7542, GNorm = 4.8751, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.354734
Epoch 15
Validation binary_cross_entropy = 0.389184
Epoch 16
Validation binary_cross_entropy = 0.403043
Epoch 17
Validation binary_cross_entropy = 0.384583
Epoch 18
Validation binary_cross_entropy = 0.386472
Epoch 19
Loss = 2.6417e-01, PNorm = 31.8162, GNorm = 5.6805, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.409624
Epoch 20
Validation binary_cross_entropy = 0.472503
Epoch 21
Validation binary_cross_entropy = 0.566765
Epoch 22
Validation binary_cross_entropy = 0.395401
Epoch 23
Validation binary_cross_entropy = 0.405448
Epoch 24
Loss = 2.1138e-01, PNorm = 31.8656, GNorm = 3.9538, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.401588
Epoch 25
Validation binary_cross_entropy = 0.456899
Epoch 26
Validation binary_cross_entropy = 0.392407
Epoch 27
Validation binary_cross_entropy = 0.397619
Epoch 28
Validation binary_cross_entropy = 0.406511
Epoch 29
Loss = 7.6526e-02, PNorm = 31.9125, GNorm = 1.4266, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.454627
Epoch 30
Validation binary_cross_entropy = 0.427443
Epoch 31
Validation binary_cross_entropy = 0.438836
Epoch 32
Validation binary_cross_entropy = 0.457467
Epoch 33
Validation binary_cross_entropy = 0.473626
Epoch 34
Loss = 1.2211e-01, PNorm = 31.9548, GNorm = 2.7547, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.562646
Epoch 35
Validation binary_cross_entropy = 0.500584
Epoch 36
Validation binary_cross_entropy = 0.508408
Epoch 37
Validation binary_cross_entropy = 0.491787
Epoch 38
Validation binary_cross_entropy = 0.480980
Epoch 39
Loss = 1.6945e-01, PNorm = 31.9944, GNorm = 4.1216, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.479329
Epoch 40
Validation binary_cross_entropy = 0.471219
Epoch 41
Validation binary_cross_entropy = 0.483982
Epoch 42
Validation binary_cross_entropy = 0.489303
Epoch 43
Validation binary_cross_entropy = 0.460287
Epoch 44
Loss = 9.6877e-02, PNorm = 32.0317, GNorm = 2.1948, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.431943
Epoch 45
Validation binary_cross_entropy = 0.429568
Epoch 46
Validation binary_cross_entropy = 0.447353
Epoch 47
Validation binary_cross_entropy = 0.527641
Epoch 48
Validation binary_cross_entropy = 0.481043
Epoch 49
Loss = 8.5093e-02, PNorm = 32.0696, GNorm = 3.6212, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.469943
Epoch 50
Validation binary_cross_entropy = 0.473616
Epoch 51
Validation binary_cross_entropy = 0.493640
Epoch 52
Validation binary_cross_entropy = 0.515261
Epoch 53
Validation binary_cross_entropy = 0.489360
Epoch 54
Loss = 2.2070e-01, PNorm = 32.1044, GNorm = 4.9654, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.484921
Epoch 55
Validation binary_cross_entropy = 0.487680
Epoch 56
Validation binary_cross_entropy = 0.494536
Epoch 57
Validation binary_cross_entropy = 0.505953
Epoch 58
Validation binary_cross_entropy = 0.515829
Epoch 59
Loss = 2.5123e-02, PNorm = 32.1328, GNorm = 0.5903, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.524281
Epoch 60
Validation binary_cross_entropy = 0.526988
Epoch 61
Validation binary_cross_entropy = 0.530855
Epoch 62
Validation binary_cross_entropy = 0.537843
Epoch 63
Validation binary_cross_entropy = 0.549379
Epoch 64
Loss = 4.0233e-02, PNorm = 32.1569, GNorm = 0.9738, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.547729
Epoch 65
Validation binary_cross_entropy = 0.550691
Epoch 66
Validation binary_cross_entropy = 0.552915
Epoch 67
Validation binary_cross_entropy = 0.558408
Epoch 68
Validation binary_cross_entropy = 0.538018
Epoch 69
Loss = 1.0448e-01, PNorm = 32.1807, GNorm = 3.4023, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.539109
Epoch 70
Validation binary_cross_entropy = 0.557795
Epoch 71
Validation binary_cross_entropy = 0.533426
Epoch 72
Validation binary_cross_entropy = 0.491701
Epoch 73
Validation binary_cross_entropy = 0.481036
Epoch 74
Loss = 1.0082e-01, PNorm = 32.2083, GNorm = 2.2983, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.491990
Epoch 75
Validation binary_cross_entropy = 0.521542
Epoch 76
Validation binary_cross_entropy = 0.560801
Epoch 77
Validation binary_cross_entropy = 0.579566
Epoch 78
Validation binary_cross_entropy = 0.600447
Epoch 79
Loss = 2.5527e-02, PNorm = 32.2379, GNorm = 0.4091, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601859
Epoch 80
Validation binary_cross_entropy = 0.614207
Epoch 81
Validation binary_cross_entropy = 0.631323
Epoch 82
Validation binary_cross_entropy = 0.646630
Epoch 83
Validation binary_cross_entropy = 0.642422
Epoch 84
Loss = 6.2074e-02, PNorm = 32.2673, GNorm = 3.6631, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.614487
Epoch 85
Validation binary_cross_entropy = 0.561924
Epoch 86
Validation binary_cross_entropy = 0.550616
Epoch 87
Validation binary_cross_entropy = 0.580334
Epoch 88
Validation binary_cross_entropy = 0.587862
Epoch 89
Loss = 3.9828e-02, PNorm = 32.3010, GNorm = 1.9294, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.587662
Epoch 90
Validation binary_cross_entropy = 0.567792
Epoch 91
Validation binary_cross_entropy = 0.570268
Epoch 92
Validation binary_cross_entropy = 0.628948
Epoch 93
Validation binary_cross_entropy = 0.669790
Epoch 94
Loss = 4.9386e-02, PNorm = 32.3306, GNorm = 2.2217, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.608086
Epoch 95
Validation binary_cross_entropy = 0.583380
Epoch 96
Validation binary_cross_entropy = 0.590414
Epoch 97
Validation binary_cross_entropy = 0.596279
Epoch 98
Validation binary_cross_entropy = 0.626397
Epoch 99
Loss = 6.3464e-02, PNorm = 32.3565, GNorm = 3.6590, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.654941
Epoch 100
Validation binary_cross_entropy = 0.653675
Epoch 101
Validation binary_cross_entropy = 0.640287
Epoch 102
Validation binary_cross_entropy = 0.619266
Epoch 103
Validation binary_cross_entropy = 0.618424
Epoch 104
Loss = 6.4208e-02, PNorm = 32.3819, GNorm = 4.0739, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.641109
Epoch 105
Validation binary_cross_entropy = 0.671874
Epoch 106
Validation binary_cross_entropy = 0.654183
Epoch 107
Validation binary_cross_entropy = 0.696812
Epoch 108
Validation binary_cross_entropy = 0.735134
Epoch 109
Loss = 5.2189e-02, PNorm = 32.4078, GNorm = 1.6231, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.754912
Epoch 110
Validation binary_cross_entropy = 0.785789
Epoch 111
Validation binary_cross_entropy = 0.787344
Epoch 112
Validation binary_cross_entropy = 0.708197
Epoch 113
Validation binary_cross_entropy = 0.661354
Epoch 114
Loss = 5.7366e-02, PNorm = 32.4361, GNorm = 1.3362, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.652214
Epoch 115
Validation binary_cross_entropy = 0.614795
Epoch 116
Validation binary_cross_entropy = 0.601197
Epoch 117
Validation binary_cross_entropy = 0.626370
Epoch 118
Validation binary_cross_entropy = 0.670822
Epoch 119
Loss = 5.3717e-02, PNorm = 32.4653, GNorm = 2.3039, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.631052
Epoch 120
Validation binary_cross_entropy = 0.614635
Epoch 121
Validation binary_cross_entropy = 0.627854
Epoch 122
Validation binary_cross_entropy = 0.646584
Epoch 123
Validation binary_cross_entropy = 0.694823
Epoch 124
Loss = 4.3830e-02, PNorm = 32.4919, GNorm = 2.5172, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.703518
Epoch 125
Validation binary_cross_entropy = 0.699271
Epoch 126
Validation binary_cross_entropy = 0.700171
Epoch 127
Validation binary_cross_entropy = 0.705154
Epoch 128
Validation binary_cross_entropy = 0.698673
Epoch 129
Loss = 4.9860e-02, PNorm = 32.5154, GNorm = 0.6879, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.731870
Epoch 130
Validation binary_cross_entropy = 0.755007
Epoch 131
Validation binary_cross_entropy = 0.757526
Epoch 132
Validation binary_cross_entropy = 0.739537
Epoch 133
Validation binary_cross_entropy = 0.710585
Epoch 134
Loss = 1.6651e-02, PNorm = 32.5358, GNorm = 0.8730, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.671115
Epoch 135
Validation binary_cross_entropy = 0.659665
Epoch 136
Validation binary_cross_entropy = 0.648511
Epoch 137
Validation binary_cross_entropy = 0.680330
Epoch 138
Validation binary_cross_entropy = 0.768991
Epoch 139
Loss = 5.2370e-02, PNorm = 32.5547, GNorm = 2.9543, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.790903
Epoch 140
Validation binary_cross_entropy = 0.726323
Epoch 141
Validation binary_cross_entropy = 0.679979
Epoch 142
Validation binary_cross_entropy = 0.698490
Epoch 143
Validation binary_cross_entropy = 0.735598
Epoch 144
Loss = 2.1502e-02, PNorm = 32.5741, GNorm = 1.8577, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.734130
Epoch 145
Validation binary_cross_entropy = 0.681340
Epoch 146
Validation binary_cross_entropy = 0.691859
Epoch 147
Validation binary_cross_entropy = 0.702060
Epoch 148
Validation binary_cross_entropy = 0.681219
Epoch 149
Loss = 5.7250e-02, PNorm = 32.5952, GNorm = 2.7399, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.698381
Epoch 150
Validation binary_cross_entropy = 0.734070
Epoch 151
Validation binary_cross_entropy = 0.716403
Epoch 152
Validation binary_cross_entropy = 0.742745
Epoch 153
Validation binary_cross_entropy = 0.738003
Epoch 154
Loss = 2.6229e-02, PNorm = 32.6156, GNorm = 1.0129, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.702246
Epoch 155
Validation binary_cross_entropy = 0.672171
Epoch 156
Validation binary_cross_entropy = 0.650511
Epoch 157
Validation binary_cross_entropy = 0.636439
Epoch 158
Validation binary_cross_entropy = 0.632556
Epoch 159
Loss = 2.5309e-02, PNorm = 32.6435, GNorm = 1.6927, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.643990
Epoch 160
Validation binary_cross_entropy = 0.683712
Epoch 161
Validation binary_cross_entropy = 0.722887
Epoch 162
Validation binary_cross_entropy = 0.779697
Epoch 163
Validation binary_cross_entropy = 0.714033
Epoch 164
Loss = 1.1855e-02, PNorm = 32.6688, GNorm = 0.9012, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.652728
Epoch 165
Validation binary_cross_entropy = 0.655451
Epoch 166
Validation binary_cross_entropy = 0.678363
Epoch 167
Validation binary_cross_entropy = 0.700314
Epoch 168
Validation binary_cross_entropy = 0.760633
Epoch 169
Loss = 6.2375e-03, PNorm = 32.6951, GNorm = 0.3536, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.830656
Epoch 170
Validation binary_cross_entropy = 0.841353
Epoch 171
Validation binary_cross_entropy = 0.795307
Epoch 172
Validation binary_cross_entropy = 0.735984
Epoch 173
Validation binary_cross_entropy = 0.712414
Epoch 174
Loss = 5.2432e-02, PNorm = 32.7162, GNorm = 2.0109, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.710721
Epoch 175
Validation binary_cross_entropy = 0.749677
Epoch 176
Validation binary_cross_entropy = 0.802833
Epoch 177
Validation binary_cross_entropy = 0.830820
Epoch 178
Validation binary_cross_entropy = 0.772721
Epoch 179
Loss = 1.8268e-02, PNorm = 32.7322, GNorm = 1.8873, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.694716
Epoch 180
Validation binary_cross_entropy = 0.666102
Epoch 181
Validation binary_cross_entropy = 0.680727
Epoch 182
Validation binary_cross_entropy = 0.712847
Epoch 183
Validation binary_cross_entropy = 0.750757
Epoch 184
Loss = 4.1640e-03, PNorm = 32.7480, GNorm = 0.3231, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.791098
Epoch 185
Validation binary_cross_entropy = 0.821570
Epoch 186
Validation binary_cross_entropy = 0.836704
Epoch 187
Validation binary_cross_entropy = 0.834756
Epoch 188
Validation binary_cross_entropy = 0.804851
Epoch 189
Loss = 6.5496e-03, PNorm = 32.7597, GNorm = 0.1130, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.769790
Epoch 190
Validation binary_cross_entropy = 0.742018
Epoch 191
Validation binary_cross_entropy = 0.731426
Epoch 192
Validation binary_cross_entropy = 0.739605
Epoch 193
Validation binary_cross_entropy = 0.756758
Epoch 194
Loss = 2.6102e-02, PNorm = 32.7701, GNorm = 2.8061, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.760506
Epoch 195
Validation binary_cross_entropy = 0.752607
Epoch 196
Validation binary_cross_entropy = 0.744792
Epoch 197
Validation binary_cross_entropy = 0.738861
Epoch 198
Validation binary_cross_entropy = 0.731540
Epoch 199
Loss = 2.3101e-03, PNorm = 32.7780, GNorm = 0.1073, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.726373
Epoch 200
Validation binary_cross_entropy = 0.729816
Epoch 201
Validation binary_cross_entropy = 0.749725
Epoch 202
Validation binary_cross_entropy = 0.794547
Epoch 203
Validation binary_cross_entropy = 0.841841
Epoch 204
Loss = 4.1513e-02, PNorm = 32.7854, GNorm = 0.7614, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.837292
Epoch 205
Validation binary_cross_entropy = 0.798047
Epoch 206
Validation binary_cross_entropy = 0.777455
Epoch 207
Validation binary_cross_entropy = 0.757611
Epoch 208
Validation binary_cross_entropy = 0.744024
Epoch 209
Loss = 9.5056e-03, PNorm = 32.7942, GNorm = 0.4451, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.735197
Epoch 210
Validation binary_cross_entropy = 0.733757
Epoch 211
Validation binary_cross_entropy = 0.754387
Epoch 212
Validation binary_cross_entropy = 0.773438
Epoch 213
Validation binary_cross_entropy = 0.802673
Epoch 214
Loss = 2.4301e-02, PNorm = 32.8037, GNorm = 0.0530, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.823084
Epoch 215
Validation binary_cross_entropy = 0.833119
Epoch 216
Validation binary_cross_entropy = 0.830474
Epoch 217
Validation binary_cross_entropy = 0.821568
Epoch 218
Validation binary_cross_entropy = 0.812929
Epoch 219
Loss = 6.5005e-03, PNorm = 32.8115, GNorm = 0.0699, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.806554
Epoch 220
Validation binary_cross_entropy = 0.837027
Epoch 221
Validation binary_cross_entropy = 0.882459
Epoch 222
Validation binary_cross_entropy = 0.896499
Epoch 223
Validation binary_cross_entropy = 0.862428
Epoch 224
Loss = 1.0218e-02, PNorm = 32.8224, GNorm = 0.4524, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.845401
Epoch 225
Validation binary_cross_entropy = 0.831579
Epoch 226
Validation binary_cross_entropy = 0.820730
Epoch 227
Validation binary_cross_entropy = 0.818800
Epoch 228
Validation binary_cross_entropy = 0.816680
Epoch 229
Loss = 1.9928e-03, PNorm = 32.8317, GNorm = 0.1016, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.814638
Epoch 230
Validation binary_cross_entropy = 0.831805
Epoch 231
Validation binary_cross_entropy = 0.867277
Epoch 232
Validation binary_cross_entropy = 0.891678
Epoch 233
Validation binary_cross_entropy = 0.865694
Epoch 234
Loss = 2.5529e-03, PNorm = 32.8387, GNorm = 0.1797, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.845799
Epoch 235
Validation binary_cross_entropy = 0.830329
Epoch 236
Validation binary_cross_entropy = 0.818645
Epoch 237
Validation binary_cross_entropy = 0.799749
Epoch 238
Validation binary_cross_entropy = 0.792060
Epoch 239
Loss = 4.3277e-02, PNorm = 32.8469, GNorm = 1.2157, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.808870
Epoch 240
Validation binary_cross_entropy = 0.845465
Epoch 241
Validation binary_cross_entropy = 0.884966
Epoch 242
Validation binary_cross_entropy = 0.917692
Epoch 243
Validation binary_cross_entropy = 0.936712
Epoch 244
Loss = 2.3959e-02, PNorm = 32.8582, GNorm = 2.4156, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.938160
Epoch 245
Validation binary_cross_entropy = 0.957587
Epoch 246
Validation binary_cross_entropy = 0.929971
Epoch 247
Validation binary_cross_entropy = 0.836686
Epoch 248
Validation binary_cross_entropy = 0.748829
Epoch 249
Loss = 5.7794e-03, PNorm = 32.8707, GNorm = 0.7859, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.717754
Epoch 250
Validation binary_cross_entropy = 0.724278
Epoch 251
Validation binary_cross_entropy = 0.762299
Epoch 252
Validation binary_cross_entropy = 0.814078
Epoch 253
Validation binary_cross_entropy = 0.834979
Epoch 254
Loss = 1.8218e-02, PNorm = 32.8858, GNorm = 0.7883, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.816922
Epoch 255
Validation binary_cross_entropy = 0.790447
Epoch 256
Validation binary_cross_entropy = 0.767599
Epoch 257
Validation binary_cross_entropy = 0.751597
Epoch 258
Validation binary_cross_entropy = 0.738590
Epoch 259
Loss = 6.2928e-03, PNorm = 32.8999, GNorm = 0.0528, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.732059
Epoch 260
Validation binary_cross_entropy = 0.729250
Epoch 261
Validation binary_cross_entropy = 0.734079
Epoch 262
Validation binary_cross_entropy = 0.743189
Epoch 263
Validation binary_cross_entropy = 0.751228
Epoch 264
Loss = 1.1049e-02, PNorm = 32.9102, GNorm = 0.1886, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.764685
Epoch 265
Validation binary_cross_entropy = 0.778863
Epoch 266
Validation binary_cross_entropy = 0.791543
Epoch 267
Validation binary_cross_entropy = 0.796558
Epoch 268
Validation binary_cross_entropy = 0.778025
Epoch 269
Loss = 4.2981e-03, PNorm = 32.9193, GNorm = 0.3400, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.774961
Epoch 270
Validation binary_cross_entropy = 0.782422
Epoch 271
Validation binary_cross_entropy = 0.794312
Epoch 272
Validation binary_cross_entropy = 0.810614
Epoch 273
Validation binary_cross_entropy = 0.830196
Epoch 274
Loss = 1.0687e-03, PNorm = 32.9257, GNorm = 0.0331, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.847421
Epoch 275
Validation binary_cross_entropy = 0.860710
Epoch 276
Validation binary_cross_entropy = 0.868172
Epoch 277
Validation binary_cross_entropy = 0.871567
Epoch 278
Validation binary_cross_entropy = 0.871538
Epoch 279
Loss = 6.0586e-03, PNorm = 32.9302, GNorm = 0.0516, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.872174
Epoch 280
Validation binary_cross_entropy = 0.873125
Epoch 281
Validation binary_cross_entropy = 0.884933
Epoch 282
Validation binary_cross_entropy = 0.900022
Epoch 283
Validation binary_cross_entropy = 0.910643
Epoch 284
Loss = 8.2487e-03, PNorm = 32.9350, GNorm = 1.2924, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.905547
Epoch 285
Validation binary_cross_entropy = 0.896105
Epoch 286
Validation binary_cross_entropy = 0.888648
Epoch 287
Validation binary_cross_entropy = 0.879540
Epoch 288
Validation binary_cross_entropy = 0.874755
Epoch 289
Loss = 1.0626e-03, PNorm = 32.9406, GNorm = 0.0528, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.873863
Epoch 290
Validation binary_cross_entropy = 0.877360
Epoch 291
Validation binary_cross_entropy = 0.886290
Epoch 292
Validation binary_cross_entropy = 0.881270
Epoch 293
Validation binary_cross_entropy = 0.882768
Epoch 294
Loss = 1.0801e-03, PNorm = 32.9464, GNorm = 0.0858, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.885644
Epoch 295
Validation binary_cross_entropy = 0.891159
Epoch 296
Validation binary_cross_entropy = 0.906022
Epoch 297
Validation binary_cross_entropy = 0.911608
Epoch 298
Validation binary_cross_entropy = 0.911480
Epoch 299
Loss = 9.3900e-03, PNorm = 32.9525, GNorm = 0.0288, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.904510
Model 0 best validation binary_cross_entropy = 0.303287 on epoch 1
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.244644
Ensemble test binary_cross_entropy = 0.244644
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.302586
Epoch 1
Validation binary_cross_entropy = 0.336095
Epoch 2
Validation binary_cross_entropy = 0.983182
Epoch 3
Validation binary_cross_entropy = 0.384951
Epoch 4
Loss = 6.5157e-01, PNorm = 31.6234, GNorm = 10.4304, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.300893
Epoch 5
Validation binary_cross_entropy = 0.449563
Epoch 6
Validation binary_cross_entropy = 0.440671
Epoch 7
Validation binary_cross_entropy = 0.408716
Epoch 8
Validation binary_cross_entropy = 0.424849
Epoch 9
Loss = 4.0301e-01, PNorm = 31.6778, GNorm = 6.6651, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.421222
Epoch 10
Validation binary_cross_entropy = 0.372895
Epoch 11
Validation binary_cross_entropy = 0.349835
Epoch 12
Validation binary_cross_entropy = 0.537450
Epoch 13
Validation binary_cross_entropy = 0.378083
Epoch 14
Loss = 3.4763e-01, PNorm = 31.7471, GNorm = 7.7848, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.363755
Epoch 15
Validation binary_cross_entropy = 0.523671
Epoch 16
Validation binary_cross_entropy = 0.612532
Epoch 17
Validation binary_cross_entropy = 0.484908
Epoch 18
Validation binary_cross_entropy = 0.464712
Epoch 19
Loss = 1.5574e-01, PNorm = 31.8091, GNorm = 2.8660, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.491154
Epoch 20
Validation binary_cross_entropy = 0.453934
Epoch 21
Validation binary_cross_entropy = 0.370938
Epoch 22
Validation binary_cross_entropy = 0.376644
Epoch 23
Validation binary_cross_entropy = 0.408917
Epoch 24
Loss = 1.2811e-01, PNorm = 31.8627, GNorm = 4.1112, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.463805
Epoch 25
Validation binary_cross_entropy = 0.446222
Epoch 26
Validation binary_cross_entropy = 0.472013
Epoch 27
Validation binary_cross_entropy = 0.480418
Epoch 28
Validation binary_cross_entropy = 0.482840
Epoch 29
Loss = 2.4552e-01, PNorm = 31.9101, GNorm = 3.6442, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.522773
Epoch 30
Validation binary_cross_entropy = 0.502394
Epoch 31
Validation binary_cross_entropy = 0.577679
Epoch 32
Validation binary_cross_entropy = 0.425061
Epoch 33
Validation binary_cross_entropy = 0.461264
Epoch 34
Loss = 1.4552e-01, PNorm = 31.9529, GNorm = 4.1074, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.460722
Epoch 35
Validation binary_cross_entropy = 0.427705
Epoch 36
Validation binary_cross_entropy = 0.421671
Epoch 37
Validation binary_cross_entropy = 0.554005
Epoch 38
Validation binary_cross_entropy = 0.567684
Epoch 39
Loss = 1.1666e-01, PNorm = 31.9951, GNorm = 2.5491, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.499432
Epoch 40
Validation binary_cross_entropy = 0.522771
Epoch 41
Validation binary_cross_entropy = 0.511869
Epoch 42
Validation binary_cross_entropy = 0.566130
Epoch 43
Validation binary_cross_entropy = 0.513527
Epoch 44
Loss = 6.4883e-02, PNorm = 32.0370, GNorm = 1.8292, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.535052
Epoch 45
Validation binary_cross_entropy = 0.556048
Epoch 46
Validation binary_cross_entropy = 0.559965
Epoch 47
Validation binary_cross_entropy = 0.527685
Epoch 48
Validation binary_cross_entropy = 0.527882
Epoch 49
Loss = 1.2220e-01, PNorm = 32.0770, GNorm = 1.0292, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.515152
Epoch 50
Validation binary_cross_entropy = 0.514067
Epoch 51
Validation binary_cross_entropy = 0.517629
Epoch 52
Validation binary_cross_entropy = 0.520688
Epoch 53
Validation binary_cross_entropy = 0.535705
Epoch 54
Loss = 4.6410e-02, PNorm = 32.1137, GNorm = 1.9048, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.538030
Epoch 55
Validation binary_cross_entropy = 0.549765
Epoch 56
Validation binary_cross_entropy = 0.569503
Epoch 57
Validation binary_cross_entropy = 0.596427
Epoch 58
Validation binary_cross_entropy = 0.572605
Epoch 59
Loss = 7.0378e-02, PNorm = 32.1484, GNorm = 3.2901, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.519531
Epoch 60
Validation binary_cross_entropy = 0.506863
Epoch 61
Validation binary_cross_entropy = 0.514800
Epoch 62
Validation binary_cross_entropy = 0.540611
Epoch 63
Validation binary_cross_entropy = 0.539460
Epoch 64
Loss = 1.0804e-01, PNorm = 32.1813, GNorm = 2.6011, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.542652
Epoch 65
Validation binary_cross_entropy = 0.557980
Epoch 66
Validation binary_cross_entropy = 0.566350
Epoch 67
Validation binary_cross_entropy = 0.588091
Epoch 68
Validation binary_cross_entropy = 0.587482
Epoch 69
Loss = 5.4360e-02, PNorm = 32.2080, GNorm = 3.6961, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.555104
Epoch 70
Validation binary_cross_entropy = 0.523101
Epoch 71
Validation binary_cross_entropy = 0.533924
Epoch 72
Validation binary_cross_entropy = 0.557801
Epoch 73
Validation binary_cross_entropy = 0.618309
Epoch 74
Loss = 4.8688e-02, PNorm = 32.2332, GNorm = 2.0544, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.649658
Epoch 75
Validation binary_cross_entropy = 0.620079
Epoch 76
Validation binary_cross_entropy = 0.592179
Epoch 77
Validation binary_cross_entropy = 0.561583
Epoch 78
Validation binary_cross_entropy = 0.551965
Epoch 79
Loss = 5.1627e-02, PNorm = 32.2583, GNorm = 0.5928, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.593466
Epoch 80
Validation binary_cross_entropy = 0.597020
Epoch 81
Validation binary_cross_entropy = 0.617423
Epoch 82
Validation binary_cross_entropy = 0.680322
Epoch 83
Validation binary_cross_entropy = 0.661855
Epoch 84
Loss = 1.5044e-02, PNorm = 32.2854, GNorm = 0.3192, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.685420
Epoch 85
Validation binary_cross_entropy = 0.725827
Epoch 86
Validation binary_cross_entropy = 0.672437
Epoch 87
Validation binary_cross_entropy = 0.684716
Epoch 88
Validation binary_cross_entropy = 0.705211
Epoch 89
Loss = 3.4456e-02, PNorm = 32.3121, GNorm = 2.4918, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.666172
Epoch 90
Validation binary_cross_entropy = 0.582048
Epoch 91
Validation binary_cross_entropy = 0.559188
Epoch 92
Validation binary_cross_entropy = 0.560377
Epoch 93
Validation binary_cross_entropy = 0.589934
Epoch 94
Loss = 8.4281e-03, PNorm = 32.3378, GNorm = 0.2259, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.637792
Epoch 95
Validation binary_cross_entropy = 0.682507
Epoch 96
Validation binary_cross_entropy = 0.646376
Epoch 97
Validation binary_cross_entropy = 0.608956
Epoch 98
Validation binary_cross_entropy = 0.600248
Epoch 99
Loss = 3.9547e-02, PNorm = 32.3634, GNorm = 2.5644, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.588507
Epoch 100
Validation binary_cross_entropy = 0.597738
Epoch 101
Validation binary_cross_entropy = 0.658723
Epoch 102
Validation binary_cross_entropy = 0.690098
Epoch 103
Validation binary_cross_entropy = 0.728030
Epoch 104
Loss = 1.0505e-02, PNorm = 32.3898, GNorm = 0.7196, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.753982
Epoch 105
Validation binary_cross_entropy = 0.742036
Epoch 106
Validation binary_cross_entropy = 0.712113
Epoch 107
Validation binary_cross_entropy = 0.644337
Epoch 108
Validation binary_cross_entropy = 0.605847
Epoch 109
Loss = 2.6668e-02, PNorm = 32.4120, GNorm = 1.4421, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.595975
Epoch 110
Validation binary_cross_entropy = 0.633025
Epoch 111
Validation binary_cross_entropy = 0.733580
Epoch 112
Validation binary_cross_entropy = 0.663931
Epoch 113
Validation binary_cross_entropy = 0.603175
Epoch 114
Loss = 7.0014e-02, PNorm = 32.4375, GNorm = 5.3869, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.616603
Epoch 115
Validation binary_cross_entropy = 0.602146
Epoch 116
Validation binary_cross_entropy = 0.642302
Epoch 117
Validation binary_cross_entropy = 0.658292
Epoch 118
Validation binary_cross_entropy = 0.641593
Epoch 119
Loss = 9.0736e-02, PNorm = 32.4770, GNorm = 1.7058, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.650492
Epoch 120
Validation binary_cross_entropy = 0.658962
Epoch 121
Validation binary_cross_entropy = 0.700409
Epoch 122
Validation binary_cross_entropy = 0.742613
Epoch 123
Validation binary_cross_entropy = 0.714301
Epoch 124
Loss = 1.3139e-02, PNorm = 32.5109, GNorm = 0.8282, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.663679
Epoch 125
Validation binary_cross_entropy = 0.668569
Epoch 126
Validation binary_cross_entropy = 0.698947
Epoch 127
Validation binary_cross_entropy = 0.722104
Epoch 128
Validation binary_cross_entropy = 0.723467
Epoch 129
Loss = 5.1220e-02, PNorm = 32.5363, GNorm = 2.2443, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.682112
Epoch 130
Validation binary_cross_entropy = 0.662970
Epoch 131
Validation binary_cross_entropy = 0.680142
Epoch 132
Validation binary_cross_entropy = 0.705826
Epoch 133
Validation binary_cross_entropy = 0.673941
Epoch 134
Loss = 3.4330e-02, PNorm = 32.5616, GNorm = 2.0659, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.626600
Epoch 135
Validation binary_cross_entropy = 0.602886
Epoch 136
Validation binary_cross_entropy = 0.597643
Epoch 137
Validation binary_cross_entropy = 0.642636
Epoch 138
Validation binary_cross_entropy = 0.706969
Epoch 139
Loss = 4.5230e-02, PNorm = 32.5843, GNorm = 0.5043, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.794108
Epoch 140
Validation binary_cross_entropy = 0.803846
Epoch 141
Validation binary_cross_entropy = 0.776429
Epoch 142
Validation binary_cross_entropy = 0.738513
Epoch 143
Validation binary_cross_entropy = 0.718009
Epoch 144
Loss = 5.9700e-03, PNorm = 32.6022, GNorm = 0.3351, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.713528
Epoch 145
Validation binary_cross_entropy = 0.729098
Epoch 146
Validation binary_cross_entropy = 0.751606
Epoch 147
Validation binary_cross_entropy = 0.785956
Epoch 148
Validation binary_cross_entropy = 0.834863
Epoch 149
Loss = 2.3083e-02, PNorm = 32.6172, GNorm = 1.4326, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.863058
Epoch 150
Validation binary_cross_entropy = 0.868900
Epoch 151
Validation binary_cross_entropy = 0.843412
Epoch 152
Validation binary_cross_entropy = 0.800436
Epoch 153
Validation binary_cross_entropy = 0.740732
Epoch 154
Loss = 1.5346e-02, PNorm = 32.6329, GNorm = 1.9255, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.681612
Epoch 155
Validation binary_cross_entropy = 0.688667
Epoch 156
Validation binary_cross_entropy = 0.691796
Epoch 157
Validation binary_cross_entropy = 0.736923
Epoch 158
Validation binary_cross_entropy = 0.778498
Epoch 159
Loss = 1.2517e-02, PNorm = 32.6491, GNorm = 1.4821, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.805512
Epoch 160
Validation binary_cross_entropy = 0.752570
Epoch 161
Validation binary_cross_entropy = 0.723078
Epoch 162
Validation binary_cross_entropy = 0.710097
Epoch 163
Validation binary_cross_entropy = 0.702101
Epoch 164
Loss = 3.2074e-02, PNorm = 32.6624, GNorm = 1.8582, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.705760
Epoch 165
Validation binary_cross_entropy = 0.722071
Epoch 166
Validation binary_cross_entropy = 0.768162
Epoch 167
Validation binary_cross_entropy = 0.836825
Epoch 168
Validation binary_cross_entropy = 0.861837
Epoch 169
Loss = 2.6469e-02, PNorm = 32.6770, GNorm = 1.6545, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.833045
Epoch 170
Validation binary_cross_entropy = 0.775048
Epoch 171
Validation binary_cross_entropy = 0.764140
Epoch 172
Validation binary_cross_entropy = 0.792826
Epoch 173
Validation binary_cross_entropy = 0.830540
Epoch 174
Loss = 8.5331e-03, PNorm = 32.6909, GNorm = 0.7627, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.880839
Epoch 175
Validation binary_cross_entropy = 0.916300
Epoch 176
Validation binary_cross_entropy = 0.941706
Epoch 177
Validation binary_cross_entropy = 0.931037
Epoch 178
Validation binary_cross_entropy = 0.873193
Epoch 179
Loss = 5.2697e-03, PNorm = 32.7053, GNorm = 0.6882, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.835339
Epoch 180
Validation binary_cross_entropy = 0.831991
Epoch 181
Validation binary_cross_entropy = 0.829750
Epoch 182
Validation binary_cross_entropy = 0.793383
Epoch 183
Validation binary_cross_entropy = 0.762971
Epoch 184
Loss = 2.7050e-02, PNorm = 32.7181, GNorm = 1.8492, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.751396
Epoch 185
Validation binary_cross_entropy = 0.793277
Epoch 186
Validation binary_cross_entropy = 0.841138
Epoch 187
Validation binary_cross_entropy = 0.858341
Epoch 188
Validation binary_cross_entropy = 0.865786
Epoch 189
Loss = 3.8740e-02, PNorm = 32.7305, GNorm = 0.1110, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.828604
Epoch 190
Validation binary_cross_entropy = 0.801880
Epoch 191
Validation binary_cross_entropy = 0.784057
Epoch 192
Validation binary_cross_entropy = 0.810737
Epoch 193
Validation binary_cross_entropy = 0.870722
Epoch 194
Loss = 1.8019e-02, PNorm = 32.7426, GNorm = 1.8661, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.901930
Epoch 195
Validation binary_cross_entropy = 0.861744
Epoch 196
Validation binary_cross_entropy = 0.839871
Epoch 197
Validation binary_cross_entropy = 0.817203
Epoch 198
Validation binary_cross_entropy = 0.801576
Epoch 199
Loss = 1.1079e-02, PNorm = 32.7537, GNorm = 0.1769, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.780971
Epoch 200
Validation binary_cross_entropy = 0.776644
Epoch 201
Validation binary_cross_entropy = 0.786832
Epoch 202
Validation binary_cross_entropy = 0.825189
Epoch 203
Validation binary_cross_entropy = 0.882992
Epoch 204
Loss = 1.7346e-02, PNorm = 32.7669, GNorm = 0.3440, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.941392
Epoch 205
Validation binary_cross_entropy = 0.963198
Epoch 206
Validation binary_cross_entropy = 0.911491
Epoch 207
Validation binary_cross_entropy = 0.865471
Epoch 208
Validation binary_cross_entropy = 0.836711
Epoch 209
Loss = 1.1782e-02, PNorm = 32.7834, GNorm = 0.1976, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.839739
Epoch 210
Validation binary_cross_entropy = 0.844503
Epoch 211
Validation binary_cross_entropy = 0.863978
Epoch 212
Validation binary_cross_entropy = 0.876918
Epoch 213
Validation binary_cross_entropy = 0.889186
Epoch 214
Loss = 3.5857e-03, PNorm = 32.7980, GNorm = 0.0605, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.873910
Epoch 215
Validation binary_cross_entropy = 0.823293
Epoch 216
Validation binary_cross_entropy = 0.784172
Epoch 217
Validation binary_cross_entropy = 0.762590
Epoch 218
Validation binary_cross_entropy = 0.723695
Epoch 219
Loss = 1.4404e-02, PNorm = 32.8092, GNorm = 0.6543, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.697353
Epoch 220
Validation binary_cross_entropy = 0.697607
Epoch 221
Validation binary_cross_entropy = 0.729323
Epoch 222
Validation binary_cross_entropy = 0.807285
Epoch 223
Validation binary_cross_entropy = 0.914980
Epoch 224
Loss = 2.8436e-03, PNorm = 32.8250, GNorm = 0.4990, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.011827
Epoch 225
Validation binary_cross_entropy = 1.077621
Epoch 226
Validation binary_cross_entropy = 1.070876
Epoch 227
Validation binary_cross_entropy = 1.021266
Epoch 228
Validation binary_cross_entropy = 0.956628
Epoch 229
Loss = 4.1410e-03, PNorm = 32.8401, GNorm = 0.0224, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.913182
Epoch 230
Validation binary_cross_entropy = 0.897650
Epoch 231
Validation binary_cross_entropy = 0.882423
Epoch 232
Validation binary_cross_entropy = 0.881626
Epoch 233
Validation binary_cross_entropy = 0.888588
Epoch 234
Loss = 2.0785e-02, PNorm = 32.8525, GNorm = 2.2722, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.915347
Epoch 235
Validation binary_cross_entropy = 0.972844
Epoch 236
Validation binary_cross_entropy = 0.989647
Epoch 237
Validation binary_cross_entropy = 0.969327
Epoch 238
Validation binary_cross_entropy = 0.932356
Epoch 239
Loss = 2.7953e-03, PNorm = 32.8625, GNorm = 0.4169, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.903640
Epoch 240
Validation binary_cross_entropy = 0.905357
Epoch 241
Validation binary_cross_entropy = 0.914956
Epoch 242
Validation binary_cross_entropy = 0.925116
Epoch 243
Validation binary_cross_entropy = 0.940844
Epoch 244
Loss = 9.7096e-03, PNorm = 32.8737, GNorm = 1.3346, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.949718
Epoch 245
Validation binary_cross_entropy = 0.928343
Epoch 246
Validation binary_cross_entropy = 0.920210
Epoch 247
Validation binary_cross_entropy = 0.917921
Epoch 248
Validation binary_cross_entropy = 0.918360
Epoch 249
Loss = 4.4156e-03, PNorm = 32.8826, GNorm = 0.6207, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.915933
Epoch 250
Validation binary_cross_entropy = 0.914307
Epoch 251
Validation binary_cross_entropy = 0.918639
Epoch 252
Validation binary_cross_entropy = 0.926939
Epoch 253
Validation binary_cross_entropy = 0.934530
Epoch 254
Loss = 6.5844e-03, PNorm = 32.8884, GNorm = 0.2339, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.951466
Epoch 255
Validation binary_cross_entropy = 0.965289
Epoch 256
Validation binary_cross_entropy = 0.974510
Epoch 257
Validation binary_cross_entropy = 0.971823
Epoch 258
Validation binary_cross_entropy = 0.959621
Epoch 259
Loss = 1.4214e-03, PNorm = 32.8931, GNorm = 0.0682, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.951001
Epoch 260
Validation binary_cross_entropy = 0.967418
Epoch 261
Validation binary_cross_entropy = 0.982287
Epoch 262
Validation binary_cross_entropy = 1.004620
Epoch 263
Validation binary_cross_entropy = 1.007955
Epoch 264
Loss = 3.6019e-03, PNorm = 32.8977, GNorm = 0.2310, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.004097
Epoch 265
Validation binary_cross_entropy = 0.992843
Epoch 266
Validation binary_cross_entropy = 0.968702
Epoch 267
Validation binary_cross_entropy = 0.963799
Epoch 268
Validation binary_cross_entropy = 0.963913
Epoch 269
Loss = 4.0082e-03, PNorm = 32.9022, GNorm = 0.2845, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.963363
Epoch 270
Validation binary_cross_entropy = 0.961583
Epoch 271
Validation binary_cross_entropy = 0.972742
Epoch 272
Validation binary_cross_entropy = 0.994445
Epoch 273
Validation binary_cross_entropy = 1.007343
Epoch 274
Loss = 2.8709e-03, PNorm = 32.9054, GNorm = 0.4252, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.013620
Epoch 275
Validation binary_cross_entropy = 0.994279
Epoch 276
Validation binary_cross_entropy = 0.946483
Epoch 277
Validation binary_cross_entropy = 0.911111
Epoch 278
Validation binary_cross_entropy = 0.887317
Epoch 279
Loss = 3.3996e-03, PNorm = 32.9094, GNorm = 0.0763, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.884363
Epoch 280
Validation binary_cross_entropy = 0.915358
Epoch 281
Validation binary_cross_entropy = 0.966533
Epoch 282
Validation binary_cross_entropy = 1.007053
Epoch 283
Validation binary_cross_entropy = 1.025183
Epoch 284
Loss = 8.5584e-03, PNorm = 32.9153, GNorm = 0.0937, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.012001
Epoch 285
Validation binary_cross_entropy = 0.993754
Epoch 286
Validation binary_cross_entropy = 0.971472
Epoch 287
Validation binary_cross_entropy = 0.917092
Epoch 288
Validation binary_cross_entropy = 0.880709
Epoch 289
Loss = 1.2211e-03, PNorm = 32.9210, GNorm = 0.0889, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.857239
Epoch 290
Validation binary_cross_entropy = 0.851705
Epoch 291
Validation binary_cross_entropy = 0.893959
Epoch 292
Validation binary_cross_entropy = 0.953681
Epoch 293
Validation binary_cross_entropy = 1.013307
Epoch 294
Loss = 3.1820e-03, PNorm = 32.9287, GNorm = 0.0990, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.059236
Epoch 295
Validation binary_cross_entropy = 1.092201
Epoch 296
Validation binary_cross_entropy = 1.106019
Epoch 297
Validation binary_cross_entropy = 1.091254
Epoch 298
Validation binary_cross_entropy = 1.057844
Epoch 299
Loss = 9.5845e-04, PNorm = 32.9348, GNorm = 0.0883, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.024480
Model 0 best validation binary_cross_entropy = 0.300893 on epoch 4
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.279091
Ensemble test binary_cross_entropy = 0.279091
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.317990
Epoch 1
Validation binary_cross_entropy = 0.325929
Epoch 2
Validation binary_cross_entropy = 0.808809
Epoch 3
Validation binary_cross_entropy = 0.427348
Epoch 4
Loss = 6.8966e-01, PNorm = 31.6229, GNorm = 5.1573, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.336841
Epoch 5
Validation binary_cross_entropy = 0.429307
Epoch 6
Validation binary_cross_entropy = 0.994567
Epoch 7
Validation binary_cross_entropy = 0.401153
Epoch 8
Validation binary_cross_entropy = 0.413958
Epoch 9
Loss = 4.4902e-01, PNorm = 31.6782, GNorm = 3.9673, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.742136
Epoch 10
Validation binary_cross_entropy = 0.591060
Epoch 11
Validation binary_cross_entropy = 0.362067
Epoch 12
Validation binary_cross_entropy = 0.374761
Epoch 13
Validation binary_cross_entropy = 0.690790
Epoch 14
Loss = 3.6549e-01, PNorm = 31.7526, GNorm = 7.5242, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.642127
Epoch 15
Validation binary_cross_entropy = 0.420999
Epoch 16
Validation binary_cross_entropy = 0.418944
Epoch 17
Validation binary_cross_entropy = 0.454150
Epoch 18
Validation binary_cross_entropy = 0.542335
Epoch 19
Loss = 2.3045e-01, PNorm = 31.8169, GNorm = 2.7332, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.457511
Epoch 20
Validation binary_cross_entropy = 0.421325
Epoch 21
Validation binary_cross_entropy = 0.437474
Epoch 22
Validation binary_cross_entropy = 0.454083
Epoch 23
Validation binary_cross_entropy = 0.434595
Epoch 24
Loss = 1.3246e-01, PNorm = 31.8707, GNorm = 2.9548, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.416565
Epoch 25
Validation binary_cross_entropy = 0.417217
Epoch 26
Validation binary_cross_entropy = 0.428007
Epoch 27
Validation binary_cross_entropy = 0.453043
Epoch 28
Validation binary_cross_entropy = 0.475608
Epoch 29
Loss = 1.2112e-01, PNorm = 31.9137, GNorm = 2.6203, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.549859
Epoch 30
Validation binary_cross_entropy = 0.561185
Epoch 31
Validation binary_cross_entropy = 0.481340
Epoch 32
Validation binary_cross_entropy = 0.435310
Epoch 33
Validation binary_cross_entropy = 0.429247
Epoch 34
Loss = 1.4757e-01, PNorm = 31.9556, GNorm = 2.9009, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.440386
Epoch 35
Validation binary_cross_entropy = 0.428242
Epoch 36
Validation binary_cross_entropy = 0.440604
Epoch 37
Validation binary_cross_entropy = 0.481573
Epoch 38
Validation binary_cross_entropy = 0.528103
Epoch 39
Loss = 1.2664e-01, PNorm = 31.9964, GNorm = 3.4849, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.500965
Epoch 40
Validation binary_cross_entropy = 0.473855
Epoch 41
Validation binary_cross_entropy = 0.479041
Epoch 42
Validation binary_cross_entropy = 0.468469
Epoch 43
Validation binary_cross_entropy = 0.464982
Epoch 44
Loss = 1.2889e-01, PNorm = 32.0365, GNorm = 4.9271, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.476475
Epoch 45
Validation binary_cross_entropy = 0.506109
Epoch 46
Validation binary_cross_entropy = 0.488878
Epoch 47
Validation binary_cross_entropy = 0.501305
Epoch 48
Validation binary_cross_entropy = 0.505512
Epoch 49
Loss = 5.7333e-02, PNorm = 32.0712, GNorm = 2.1434, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.527023
Epoch 50
Validation binary_cross_entropy = 0.563819
Epoch 51
Validation binary_cross_entropy = 0.606429
Epoch 52
Validation binary_cross_entropy = 0.619735
Epoch 53
Validation binary_cross_entropy = 0.611862
Epoch 54
Loss = 7.8524e-02, PNorm = 32.1043, GNorm = 2.0810, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.564136
Epoch 55
Validation binary_cross_entropy = 0.541685
Epoch 56
Validation binary_cross_entropy = 0.537022
Epoch 57
Validation binary_cross_entropy = 0.530596
Epoch 58
Validation binary_cross_entropy = 0.543357
Epoch 59
Loss = 7.8469e-02, PNorm = 32.1348, GNorm = 2.2100, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.574326
Epoch 60
Validation binary_cross_entropy = 0.582260
Epoch 61
Validation binary_cross_entropy = 0.546817
Epoch 62
Validation binary_cross_entropy = 0.542984
Epoch 63
Validation binary_cross_entropy = 0.549344
Epoch 64
Loss = 3.3478e-02, PNorm = 32.1596, GNorm = 1.0599, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.568160
Epoch 65
Validation binary_cross_entropy = 0.658133
Epoch 66
Validation binary_cross_entropy = 0.668788
Epoch 67
Validation binary_cross_entropy = 0.652038
Epoch 68
Validation binary_cross_entropy = 0.683694
Epoch 69
Loss = 4.3433e-02, PNorm = 32.1896, GNorm = 1.5910, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.667512
Epoch 70
Validation binary_cross_entropy = 0.665319
Epoch 71
Validation binary_cross_entropy = 0.649939
Epoch 72
Validation binary_cross_entropy = 0.587306
Epoch 73
Validation binary_cross_entropy = 0.616098
Epoch 74
Loss = 8.1108e-02, PNorm = 32.2255, GNorm = 2.0298, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.609794
Epoch 75
Validation binary_cross_entropy = 0.609011
Epoch 76
Validation binary_cross_entropy = 0.664352
Epoch 77
Validation binary_cross_entropy = 0.674472
Epoch 78
Validation binary_cross_entropy = 0.632765
Epoch 79
Loss = 6.4939e-02, PNorm = 32.2608, GNorm = 1.8249, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.652042
Epoch 80
Validation binary_cross_entropy = 0.681404
Epoch 81
Validation binary_cross_entropy = 0.677892
Epoch 82
Validation binary_cross_entropy = 0.681238
Epoch 83
Validation binary_cross_entropy = 0.658693
Epoch 84
Loss = 2.5284e-02, PNorm = 32.2983, GNorm = 0.6740, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.616239
Epoch 85
Validation binary_cross_entropy = 0.603232
Epoch 86
Validation binary_cross_entropy = 0.606607
Epoch 87
Validation binary_cross_entropy = 0.609945
Epoch 88
Validation binary_cross_entropy = 0.615673
Epoch 89
Loss = 3.9866e-02, PNorm = 32.3306, GNorm = 1.3869, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.625737
Epoch 90
Validation binary_cross_entropy = 0.630218
Epoch 91
Validation binary_cross_entropy = 0.604396
Epoch 92
Validation binary_cross_entropy = 0.615074
Epoch 93
Validation binary_cross_entropy = 0.636945
Epoch 94
Loss = 3.0899e-02, PNorm = 32.3554, GNorm = 0.6270, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.635589
Epoch 95
Validation binary_cross_entropy = 0.648357
Epoch 96
Validation binary_cross_entropy = 0.675411
Epoch 97
Validation binary_cross_entropy = 0.713329
Epoch 98
Validation binary_cross_entropy = 0.730276
Epoch 99
Loss = 1.1562e-02, PNorm = 32.3757, GNorm = 0.4834, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.729828
Epoch 100
Validation binary_cross_entropy = 0.690137
Epoch 101
Validation binary_cross_entropy = 0.678888
Epoch 102
Validation binary_cross_entropy = 0.689086
Epoch 103
Validation binary_cross_entropy = 0.724523
Epoch 104
Loss = 2.2764e-02, PNorm = 32.3958, GNorm = 1.5313, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.750543
Epoch 105
Validation binary_cross_entropy = 0.753471
Epoch 106
Validation binary_cross_entropy = 0.761664
Epoch 107
Validation binary_cross_entropy = 0.772726
Epoch 108
Validation binary_cross_entropy = 0.798379
Epoch 109
Loss = 3.5870e-02, PNorm = 32.4140, GNorm = 3.3105, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.797438
Epoch 110
Validation binary_cross_entropy = 0.713222
Epoch 111
Validation binary_cross_entropy = 0.677779
Epoch 112
Validation binary_cross_entropy = 0.697189
Epoch 113
Validation binary_cross_entropy = 0.788580
Epoch 114
Loss = 4.1686e-02, PNorm = 32.4370, GNorm = 1.5907, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.797624
Epoch 115
Validation binary_cross_entropy = 0.746081
Epoch 116
Validation binary_cross_entropy = 0.724400
Epoch 117
Validation binary_cross_entropy = 0.740119
Epoch 118
Validation binary_cross_entropy = 0.740248
Epoch 119
Loss = 5.4929e-02, PNorm = 32.4621, GNorm = 2.4201, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.763068
Epoch 120
Validation binary_cross_entropy = 0.747386
Epoch 121
Validation binary_cross_entropy = 0.761544
Epoch 122
Validation binary_cross_entropy = 0.776555
Epoch 123
Validation binary_cross_entropy = 0.772372
Epoch 124
Loss = 2.8443e-02, PNorm = 32.4872, GNorm = 0.6634, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.742882
Epoch 125
Validation binary_cross_entropy = 0.754435
Epoch 126
Validation binary_cross_entropy = 0.778974
Epoch 127
Validation binary_cross_entropy = 0.807675
Epoch 128
Validation binary_cross_entropy = 0.833702
Epoch 129
Loss = 1.1828e-01, PNorm = 32.5108, GNorm = 1.0045, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.815813
Epoch 130
Validation binary_cross_entropy = 0.798032
Epoch 131
Validation binary_cross_entropy = 0.762330
Epoch 132
Validation binary_cross_entropy = 0.747177
Epoch 133
Validation binary_cross_entropy = 0.761333
Epoch 134
Loss = 8.2491e-03, PNorm = 32.5358, GNorm = 0.8474, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.799735
Epoch 135
Validation binary_cross_entropy = 0.805021
Epoch 136
Validation binary_cross_entropy = 0.780423
Epoch 137
Validation binary_cross_entropy = 0.773025
Epoch 138
Validation binary_cross_entropy = 0.783559
Epoch 139
Loss = 4.6108e-02, PNorm = 32.5602, GNorm = 0.2758, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.779248
Epoch 140
Validation binary_cross_entropy = 0.772238
Epoch 141
Validation binary_cross_entropy = 0.847129
Epoch 142
Validation binary_cross_entropy = 0.826924
Epoch 143
Validation binary_cross_entropy = 0.751183
Epoch 144
Loss = 1.3831e-02, PNorm = 32.5923, GNorm = 0.3253, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.764681
Epoch 145
Validation binary_cross_entropy = 0.782683
Epoch 146
Validation binary_cross_entropy = 0.762211
Epoch 147
Validation binary_cross_entropy = 0.840934
Epoch 148
Validation binary_cross_entropy = 0.869190
Epoch 149
Loss = 6.8222e-02, PNorm = 32.6269, GNorm = 0.5706, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.769323
Epoch 150
Validation binary_cross_entropy = 0.731716
Epoch 151
Validation binary_cross_entropy = 0.726030
Epoch 152
Validation binary_cross_entropy = 0.725606
Epoch 153
Validation binary_cross_entropy = 0.724027
Epoch 154
Loss = 1.2809e-02, PNorm = 32.6606, GNorm = 0.8629, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.737895
Epoch 155
Validation binary_cross_entropy = 0.752472
Epoch 156
Validation binary_cross_entropy = 0.771250
Epoch 157
Validation binary_cross_entropy = 0.782594
Epoch 158
Validation binary_cross_entropy = 0.787636
Epoch 159
Loss = 4.6526e-02, PNorm = 32.6884, GNorm = 3.7699, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.801970
Epoch 160
Validation binary_cross_entropy = 0.802980
Epoch 161
Validation binary_cross_entropy = 0.785250
Epoch 162
Validation binary_cross_entropy = 0.744413
Epoch 163
Validation binary_cross_entropy = 0.714887
Epoch 164
Loss = 2.1700e-02, PNorm = 32.7084, GNorm = 1.3034, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.728910
Epoch 165
Validation binary_cross_entropy = 0.714793
Epoch 166
Validation binary_cross_entropy = 0.729216
Epoch 167
Validation binary_cross_entropy = 0.777129
Epoch 168
Validation binary_cross_entropy = 0.828800
Epoch 169
Loss = 8.1735e-02, PNorm = 32.7258, GNorm = 3.6522, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.860226
Epoch 170
Validation binary_cross_entropy = 0.819625
Epoch 171
Validation binary_cross_entropy = 0.770090
Epoch 172
Validation binary_cross_entropy = 0.745555
Epoch 173
Validation binary_cross_entropy = 0.756698
Epoch 174
Loss = 2.7716e-03, PNorm = 32.7429, GNorm = 0.2896, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.773005
Epoch 175
Validation binary_cross_entropy = 0.815128
Epoch 176
Validation binary_cross_entropy = 0.875134
Epoch 177
Validation binary_cross_entropy = 0.923866
Epoch 178
Validation binary_cross_entropy = 0.983684
Epoch 179
Loss = 2.2318e-02, PNorm = 32.7592, GNorm = 1.4431, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.998490
Epoch 180
Validation binary_cross_entropy = 0.828718
Epoch 181
Validation binary_cross_entropy = 0.680817
Epoch 182
Validation binary_cross_entropy = 0.630612
Epoch 183
Validation binary_cross_entropy = 0.623697
Epoch 184
Loss = 7.5939e-02, PNorm = 32.7869, GNorm = 4.9174, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.637195
Epoch 185
Validation binary_cross_entropy = 0.709039
Epoch 186
Validation binary_cross_entropy = 0.866136
Epoch 187
Validation binary_cross_entropy = 0.951826
Epoch 188
Validation binary_cross_entropy = 0.892765
Epoch 189
Loss = 5.9037e-02, PNorm = 32.8207, GNorm = 2.4822, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.771643
Epoch 190
Validation binary_cross_entropy = 0.763458
Epoch 191
Validation binary_cross_entropy = 0.804616
Epoch 192
Validation binary_cross_entropy = 0.777316
Epoch 193
Validation binary_cross_entropy = 0.826681
Epoch 194
Loss = 6.5474e-02, PNorm = 32.8566, GNorm = 2.7745, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.873459
Epoch 195
Validation binary_cross_entropy = 0.864141
Epoch 196
Validation binary_cross_entropy = 0.829435
Epoch 197
Validation binary_cross_entropy = 0.798506
Epoch 198
Validation binary_cross_entropy = 0.776265
Epoch 199
Loss = 1.6152e-02, PNorm = 32.8893, GNorm = 0.9232, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.766512
Epoch 200
Validation binary_cross_entropy = 0.776899
Epoch 201
Validation binary_cross_entropy = 0.813829
Epoch 202
Validation binary_cross_entropy = 0.828249
Epoch 203
Validation binary_cross_entropy = 0.753982
Epoch 204
Loss = 7.5889e-03, PNorm = 32.9107, GNorm = 0.4100, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.712056
Epoch 205
Validation binary_cross_entropy = 0.687088
Epoch 206
Validation binary_cross_entropy = 0.676383
Epoch 207
Validation binary_cross_entropy = 0.684439
Epoch 208
Validation binary_cross_entropy = 0.722858
Epoch 209
Loss = 3.4125e-02, PNorm = 32.9287, GNorm = 1.7879, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.775285
Epoch 210
Validation binary_cross_entropy = 0.828849
Epoch 211
Validation binary_cross_entropy = 0.861233
Epoch 212
Validation binary_cross_entropy = 0.846234
Epoch 213
Validation binary_cross_entropy = 0.798428
Epoch 214
Loss = 6.7971e-03, PNorm = 32.9443, GNorm = 0.4599, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.762394
Epoch 215
Validation binary_cross_entropy = 0.741454
Epoch 216
Validation binary_cross_entropy = 0.740062
Epoch 217
Validation binary_cross_entropy = 0.764789
Epoch 218
Validation binary_cross_entropy = 0.790887
Epoch 219
Loss = 3.1601e-03, PNorm = 32.9586, GNorm = 0.3255, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.815324
Epoch 220
Validation binary_cross_entropy = 0.837112
Epoch 221
Validation binary_cross_entropy = 0.854591
Epoch 222
Validation binary_cross_entropy = 0.869283
Epoch 223
Validation binary_cross_entropy = 0.870332
Epoch 224
Loss = 1.8500e-02, PNorm = 32.9696, GNorm = 1.3113, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.848267
Epoch 225
Validation binary_cross_entropy = 0.834705
Epoch 226
Validation binary_cross_entropy = 0.823270
Epoch 227
Validation binary_cross_entropy = 0.813177
Epoch 228
Validation binary_cross_entropy = 0.851455
Epoch 229
Loss = 7.9113e-03, PNorm = 32.9778, GNorm = 0.2122, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.876658
Epoch 230
Validation binary_cross_entropy = 0.892285
Epoch 231
Validation binary_cross_entropy = 0.922670
Epoch 232
Validation binary_cross_entropy = 0.931718
Epoch 233
Validation binary_cross_entropy = 0.930777
Epoch 234
Loss = 3.5713e-03, PNorm = 32.9862, GNorm = 0.1026, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.922405
Epoch 235
Validation binary_cross_entropy = 0.902994
Epoch 236
Validation binary_cross_entropy = 0.874985
Epoch 237
Validation binary_cross_entropy = 0.831117
Epoch 238
Validation binary_cross_entropy = 0.810385
Epoch 239
Loss = 3.0999e-02, PNorm = 32.9934, GNorm = 2.5709, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.825947
Epoch 240
Validation binary_cross_entropy = 0.878333
Epoch 241
Validation binary_cross_entropy = 0.938121
Epoch 242
Validation binary_cross_entropy = 1.002153
Epoch 243
Validation binary_cross_entropy = 1.002061
Epoch 244
Loss = 5.8893e-03, PNorm = 33.0072, GNorm = 0.4714, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.962871
Epoch 245
Validation binary_cross_entropy = 0.917764
Epoch 246
Validation binary_cross_entropy = 0.890966
Epoch 247
Validation binary_cross_entropy = 0.877045
Epoch 248
Validation binary_cross_entropy = 0.871933
Epoch 249
Loss = 3.0107e-03, PNorm = 33.0196, GNorm = 0.2486, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.873124
Epoch 250
Validation binary_cross_entropy = 0.887684
Epoch 251
Validation binary_cross_entropy = 0.894779
Epoch 252
Validation binary_cross_entropy = 0.888487
Epoch 253
Validation binary_cross_entropy = 0.881682
Epoch 254
Loss = 1.2051e-03, PNorm = 33.0274, GNorm = 0.0221, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.874070
Epoch 255
Validation binary_cross_entropy = 0.863745
Epoch 256
Validation binary_cross_entropy = 0.858914
Epoch 257
Validation binary_cross_entropy = 0.857965
Epoch 258
Validation binary_cross_entropy = 0.860565
Epoch 259
Loss = 4.0224e-02, PNorm = 33.0323, GNorm = 3.1877, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.882724
Epoch 260
Validation binary_cross_entropy = 0.916902
Epoch 261
Validation binary_cross_entropy = 0.956650
Epoch 262
Validation binary_cross_entropy = 0.969189
Epoch 263
Validation binary_cross_entropy = 0.942255
Epoch 264
Loss = 4.9293e-03, PNorm = 33.0384, GNorm = 0.3187, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.925144
Epoch 265
Validation binary_cross_entropy = 0.914312
Epoch 266
Validation binary_cross_entropy = 0.906697
Epoch 267
Validation binary_cross_entropy = 0.906569
Epoch 268
Validation binary_cross_entropy = 0.909473
Epoch 269
Loss = 5.3128e-02, PNorm = 33.0442, GNorm = 0.5703, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.927141
Epoch 270
Validation binary_cross_entropy = 0.961085
Epoch 271
Validation binary_cross_entropy = 0.993963
Epoch 272
Validation binary_cross_entropy = 1.013616
Epoch 273
Validation binary_cross_entropy = 1.004826
Epoch 274
Loss = 8.3385e-03, PNorm = 33.0494, GNorm = 0.0433, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.983227
Epoch 275
Validation binary_cross_entropy = 0.944315
Epoch 276
Validation binary_cross_entropy = 0.916845
Epoch 277
Validation binary_cross_entropy = 0.898692
Epoch 278
Validation binary_cross_entropy = 0.889848
Epoch 279
Loss = 2.4093e-02, PNorm = 33.0573, GNorm = 0.0581, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.904206
Epoch 280
Validation binary_cross_entropy = 0.918919
Epoch 281
Validation binary_cross_entropy = 0.949034
Epoch 282
Validation binary_cross_entropy = 1.000545
Epoch 283
Validation binary_cross_entropy = 1.039862
Epoch 284
Loss = 5.5722e-02, PNorm = 33.0688, GNorm = 0.2014, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.981890
Epoch 285
Validation binary_cross_entropy = 0.920654
Epoch 286
Validation binary_cross_entropy = 0.876970
Epoch 287
Validation binary_cross_entropy = 0.861652
Epoch 288
Validation binary_cross_entropy = 0.851793
Epoch 289
Loss = 2.6777e-03, PNorm = 33.0888, GNorm = 0.2727, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.846765
Epoch 290
Validation binary_cross_entropy = 0.849193
Epoch 291
Validation binary_cross_entropy = 0.863116
Epoch 292
Validation binary_cross_entropy = 0.884871
Epoch 293
Validation binary_cross_entropy = 0.907783
Epoch 294
Loss = 1.0589e-03, PNorm = 33.1046, GNorm = 0.1033, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.928530
Epoch 295
Validation binary_cross_entropy = 0.945600
Epoch 296
Validation binary_cross_entropy = 0.958472
Epoch 297
Validation binary_cross_entropy = 1.017697
Epoch 298
Validation binary_cross_entropy = 1.061164
Epoch 299
Loss = 4.0107e-03, PNorm = 33.1154, GNorm = 0.5093, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.074110
Model 0 best validation binary_cross_entropy = 0.317990 on epoch 0
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.320771
Ensemble test binary_cross_entropy = 0.320771
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288182
Epoch 1
Validation binary_cross_entropy = 0.283089
Epoch 2
Validation binary_cross_entropy = 0.828666
Epoch 3
Validation binary_cross_entropy = 0.705652
Epoch 4
Loss = 6.8773e-01, PNorm = 31.6242, GNorm = 7.9209, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.316085
Epoch 5
Validation binary_cross_entropy = 0.373328
Epoch 6
Validation binary_cross_entropy = 0.986213
Epoch 7
Validation binary_cross_entropy = 0.701273
Epoch 8
Validation binary_cross_entropy = 0.419734
Epoch 9
Loss = 7.1211e-01, PNorm = 31.6811, GNorm = 9.4318, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.422149
Epoch 10
Validation binary_cross_entropy = 0.907744
Epoch 11
Validation binary_cross_entropy = 0.601722
Epoch 12
Validation binary_cross_entropy = 0.453335
Epoch 13
Validation binary_cross_entropy = 0.528654
Epoch 14
Loss = 3.2482e-01, PNorm = 31.7559, GNorm = 4.3821, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.567754
Epoch 15
Validation binary_cross_entropy = 0.506934
Epoch 16
Validation binary_cross_entropy = 0.379532
Epoch 17
Validation binary_cross_entropy = 0.375121
Epoch 18
Validation binary_cross_entropy = 0.424528
Epoch 19
Loss = 1.4010e-01, PNorm = 31.8199, GNorm = 2.3482, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.450889
Epoch 20
Validation binary_cross_entropy = 0.395330
Epoch 21
Validation binary_cross_entropy = 0.391465
Epoch 22
Validation binary_cross_entropy = 0.395124
Epoch 23
Validation binary_cross_entropy = 0.422287
Epoch 24
Loss = 1.7166e-01, PNorm = 31.8719, GNorm = 3.3637, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.443416
Epoch 25
Validation binary_cross_entropy = 0.462594
Epoch 26
Validation binary_cross_entropy = 0.469582
Epoch 27
Validation binary_cross_entropy = 0.470121
Epoch 28
Validation binary_cross_entropy = 0.474304
Epoch 29
Loss = 1.7542e-01, PNorm = 31.9171, GNorm = 2.9811, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.579147
Epoch 30
Validation binary_cross_entropy = 0.490408
Epoch 31
Validation binary_cross_entropy = 0.486811
Epoch 32
Validation binary_cross_entropy = 0.456645
Epoch 33
Validation binary_cross_entropy = 0.453819
Epoch 34
Loss = 1.6940e-01, PNorm = 31.9585, GNorm = 2.9363, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.466960
Epoch 35
Validation binary_cross_entropy = 0.458908
Epoch 36
Validation binary_cross_entropy = 0.442531
Epoch 37
Validation binary_cross_entropy = 0.496137
Epoch 38
Validation binary_cross_entropy = 0.474494
Epoch 39
Loss = 1.1807e-01, PNorm = 31.9988, GNorm = 3.0485, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.473629
Epoch 40
Validation binary_cross_entropy = 0.484769
Epoch 41
Validation binary_cross_entropy = 0.504303
Epoch 42
Validation binary_cross_entropy = 0.501280
Epoch 43
Validation binary_cross_entropy = 0.491701
Epoch 44
Loss = 6.0866e-02, PNorm = 32.0364, GNorm = 0.9855, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.506160
Epoch 45
Validation binary_cross_entropy = 0.524846
Epoch 46
Validation binary_cross_entropy = 0.533975
Epoch 47
Validation binary_cross_entropy = 0.525718
Epoch 48
Validation binary_cross_entropy = 0.531333
Epoch 49
Loss = 1.1105e-01, PNorm = 32.0734, GNorm = 0.7339, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.523629
Epoch 50
Validation binary_cross_entropy = 0.537344
Epoch 51
Validation binary_cross_entropy = 0.565278
Epoch 52
Validation binary_cross_entropy = 0.585784
Epoch 53
Validation binary_cross_entropy = 0.544915
Epoch 54
Loss = 8.6704e-02, PNorm = 32.1108, GNorm = 3.0411, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.529067
Epoch 55
Validation binary_cross_entropy = 0.534740
Epoch 56
Validation binary_cross_entropy = 0.540591
Epoch 57
Validation binary_cross_entropy = 0.561352
Epoch 58
Validation binary_cross_entropy = 0.544383
Epoch 59
Loss = 5.6676e-02, PNorm = 32.1518, GNorm = 1.2561, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.506246
Epoch 60
Validation binary_cross_entropy = 0.553885
Epoch 61
Validation binary_cross_entropy = 0.606001
Epoch 62
Validation binary_cross_entropy = 0.599455
Epoch 63
Validation binary_cross_entropy = 0.546370
Epoch 64
Loss = 5.5247e-02, PNorm = 32.1917, GNorm = 2.9728, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.521154
Epoch 65
Validation binary_cross_entropy = 0.501284
Epoch 66
Validation binary_cross_entropy = 0.524739
Epoch 67
Validation binary_cross_entropy = 0.520145
Epoch 68
Validation binary_cross_entropy = 0.533114
Epoch 69
Loss = 9.1254e-02, PNorm = 32.2309, GNorm = 4.0410, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.605507
Epoch 70
Validation binary_cross_entropy = 0.576404
Epoch 71
Validation binary_cross_entropy = 0.630129
Epoch 72
Validation binary_cross_entropy = 0.680823
Epoch 73
Validation binary_cross_entropy = 0.583724
Epoch 74
Loss = 2.4029e-02, PNorm = 32.2754, GNorm = 1.6527, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.715403
Epoch 75
Validation binary_cross_entropy = 0.694523
Epoch 76
Validation binary_cross_entropy = 0.644082
Epoch 77
Validation binary_cross_entropy = 0.726033
Epoch 78
Validation binary_cross_entropy = 0.637175
Epoch 79
Loss = 6.0594e-02, PNorm = 32.3229, GNorm = 2.1055, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.582756
Epoch 80
Validation binary_cross_entropy = 0.560027
Epoch 81
Validation binary_cross_entropy = 0.554882
Epoch 82
Validation binary_cross_entropy = 0.593563
Epoch 83
Validation binary_cross_entropy = 0.609247
Epoch 84
Loss = 6.6970e-02, PNorm = 32.3649, GNorm = 1.5730, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.602128
Epoch 85
Validation binary_cross_entropy = 0.581436
Epoch 86
Validation binary_cross_entropy = 0.574290
Epoch 87
Validation binary_cross_entropy = 0.566208
Epoch 88
Validation binary_cross_entropy = 0.568638
Epoch 89
Loss = 5.9617e-02, PNorm = 32.3979, GNorm = 0.9294, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.589198
Epoch 90
Validation binary_cross_entropy = 0.596461
Epoch 91
Validation binary_cross_entropy = 0.593650
Epoch 92
Validation binary_cross_entropy = 0.595450
Epoch 93
Validation binary_cross_entropy = 0.602658
Epoch 94
Loss = 6.8072e-02, PNorm = 32.4239, GNorm = 2.0492, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.615782
Epoch 95
Validation binary_cross_entropy = 0.661427
Epoch 96
Validation binary_cross_entropy = 0.692869
Epoch 97
Validation binary_cross_entropy = 0.644795
Epoch 98
Validation binary_cross_entropy = 0.618286
Epoch 99
Loss = 3.3468e-02, PNorm = 32.4473, GNorm = 0.4251, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.604027
Epoch 100
Validation binary_cross_entropy = 0.601894
Epoch 101
Validation binary_cross_entropy = 0.607582
Epoch 102
Validation binary_cross_entropy = 0.613422
Epoch 103
Validation binary_cross_entropy = 0.632323
Epoch 104
Loss = 2.0631e-02, PNorm = 32.4705, GNorm = 1.2613, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.653862
Epoch 105
Validation binary_cross_entropy = 0.678195
Epoch 106
Validation binary_cross_entropy = 0.692831
Epoch 107
Validation binary_cross_entropy = 0.705251
Epoch 108
Validation binary_cross_entropy = 0.685132
Epoch 109
Loss = 2.5769e-02, PNorm = 32.4884, GNorm = 0.1149, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.653265
Epoch 110
Validation binary_cross_entropy = 0.641618
Epoch 111
Validation binary_cross_entropy = 0.644066
Epoch 112
Validation binary_cross_entropy = 0.669784
Epoch 113
Validation binary_cross_entropy = 0.708735
Epoch 114
Loss = 4.8827e-02, PNorm = 32.5020, GNorm = 1.6702, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.689740
Epoch 115
Validation binary_cross_entropy = 0.645633
Epoch 116
Validation binary_cross_entropy = 0.606895
Epoch 117
Validation binary_cross_entropy = 0.590290
Epoch 118
Validation binary_cross_entropy = 0.591045
Epoch 119
Loss = 4.3520e-02, PNorm = 32.5171, GNorm = 1.5969, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.607142
Epoch 120
Validation binary_cross_entropy = 0.654153
Epoch 121
Validation binary_cross_entropy = 0.701896
Epoch 122
Validation binary_cross_entropy = 0.728122
Epoch 123
Validation binary_cross_entropy = 0.712288
Epoch 124
Loss = 3.4150e-02, PNorm = 32.5330, GNorm = 2.2769, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.667841
Epoch 125
Validation binary_cross_entropy = 0.627195
Epoch 126
Validation binary_cross_entropy = 0.607820
Epoch 127
Validation binary_cross_entropy = 0.620540
Epoch 128
Validation binary_cross_entropy = 0.651686
Epoch 129
Loss = 2.1780e-02, PNorm = 32.5467, GNorm = 1.2082, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.684532
Epoch 130
Validation binary_cross_entropy = 0.737687
Epoch 131
Validation binary_cross_entropy = 0.774587
Epoch 132
Validation binary_cross_entropy = 0.761830
Epoch 133
Validation binary_cross_entropy = 0.715013
Epoch 134
Loss = 4.5029e-03, PNorm = 32.5590, GNorm = 0.2288, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.700899
Epoch 135
Validation binary_cross_entropy = 0.716542
Epoch 136
Validation binary_cross_entropy = 0.730794
Epoch 137
Validation binary_cross_entropy = 0.749220
Epoch 138
Validation binary_cross_entropy = 0.790129
Epoch 139
Loss = 2.5809e-02, PNorm = 32.5702, GNorm = 0.3956, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.818887
Epoch 140
Validation binary_cross_entropy = 0.825347
Epoch 141
Validation binary_cross_entropy = 0.812336
Epoch 142
Validation binary_cross_entropy = 0.799108
Epoch 143
Validation binary_cross_entropy = 0.783924
Epoch 144
Loss = 1.0697e-02, PNorm = 32.5826, GNorm = 0.4986, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.786796
Epoch 145
Validation binary_cross_entropy = 0.800747
Epoch 146
Validation binary_cross_entropy = 0.819048
Epoch 147
Validation binary_cross_entropy = 0.850659
Epoch 148
Validation binary_cross_entropy = 0.869033
Epoch 149
Loss = 1.0748e-02, PNorm = 32.5952, GNorm = 0.4114, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.850982
Epoch 150
Validation binary_cross_entropy = 0.824989
Epoch 151
Validation binary_cross_entropy = 0.794959
Epoch 152
Validation binary_cross_entropy = 0.780413
Epoch 153
Validation binary_cross_entropy = 0.762324
Epoch 154
Loss = 2.3065e-02, PNorm = 32.6077, GNorm = 0.1231, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.786762
Epoch 155
Validation binary_cross_entropy = 0.810506
Epoch 156
Validation binary_cross_entropy = 0.799174
Epoch 157
Validation binary_cross_entropy = 0.791610
Epoch 158
Validation binary_cross_entropy = 0.798266
Epoch 159
Loss = 8.4411e-03, PNorm = 32.6215, GNorm = 0.2338, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.814856
Epoch 160
Validation binary_cross_entropy = 0.818886
Epoch 161
Validation binary_cross_entropy = 0.826308
Epoch 162
Validation binary_cross_entropy = 0.830032
Epoch 163
Validation binary_cross_entropy = 0.824554
Epoch 164
Loss = 3.3902e-03, PNorm = 32.6330, GNorm = 0.2355, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.812094
Epoch 165
Validation binary_cross_entropy = 0.795797
Epoch 166
Validation binary_cross_entropy = 0.766208
Epoch 167
Validation binary_cross_entropy = 0.708833
Epoch 168
Validation binary_cross_entropy = 0.687211
Epoch 169
Loss = 8.0498e-02, PNorm = 32.6433, GNorm = 0.7883, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.692142
Epoch 170
Validation binary_cross_entropy = 0.716383
Epoch 171
Validation binary_cross_entropy = 0.749236
Epoch 172
Validation binary_cross_entropy = 0.820257
Epoch 173
Validation binary_cross_entropy = 0.838202
Epoch 174
Loss = 1.3805e-02, PNorm = 32.6553, GNorm = 0.7355, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.812584
Epoch 175
Validation binary_cross_entropy = 0.777358
Epoch 176
Validation binary_cross_entropy = 0.735774
Epoch 177
Validation binary_cross_entropy = 0.697797
Epoch 178
Validation binary_cross_entropy = 0.681469
Epoch 179
Loss = 1.5865e-02, PNorm = 32.6681, GNorm = 0.5197, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.677145
Epoch 180
Validation binary_cross_entropy = 0.718225
Epoch 181
Validation binary_cross_entropy = 0.764278
Epoch 182
Validation binary_cross_entropy = 0.791620
Epoch 183
Validation binary_cross_entropy = 0.764449
Epoch 184
Loss = 6.5369e-02, PNorm = 32.6805, GNorm = 3.5399, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.755336
Epoch 185
Validation binary_cross_entropy = 0.768719
Epoch 186
Validation binary_cross_entropy = 0.782051
Epoch 187
Validation binary_cross_entropy = 0.770449
Epoch 188
Validation binary_cross_entropy = 0.751786
Epoch 189
Loss = 2.6988e-02, PNorm = 32.6929, GNorm = 1.0536, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.752027
Epoch 190
Validation binary_cross_entropy = 0.754944
Epoch 191
Validation binary_cross_entropy = 0.772285
Epoch 192
Validation binary_cross_entropy = 0.819707
Epoch 193
Validation binary_cross_entropy = 0.858603
Epoch 194
Loss = 6.0121e-03, PNorm = 32.7064, GNorm = 0.5085, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.869851
Epoch 195
Validation binary_cross_entropy = 0.846994
Epoch 196
Validation binary_cross_entropy = 0.822001
Epoch 197
Validation binary_cross_entropy = 0.798106
Epoch 198
Validation binary_cross_entropy = 0.779722
Epoch 199
Loss = 2.3761e-02, PNorm = 32.7186, GNorm = 2.4411, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.775999
Epoch 200
Validation binary_cross_entropy = 0.772932
Epoch 201
Validation binary_cross_entropy = 0.777826
Epoch 202
Validation binary_cross_entropy = 0.793831
Epoch 203
Validation binary_cross_entropy = 0.808350
Epoch 204
Loss = 4.6844e-02, PNorm = 32.7308, GNorm = 0.4036, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.851805
Epoch 205
Validation binary_cross_entropy = 0.868859
Epoch 206
Validation binary_cross_entropy = 0.841635
Epoch 207
Validation binary_cross_entropy = 0.813467
Epoch 208
Validation binary_cross_entropy = 0.795215
Epoch 209
Loss = 3.0286e-03, PNorm = 32.7424, GNorm = 0.3317, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.787412
Epoch 210
Validation binary_cross_entropy = 0.790207
Epoch 211
Validation binary_cross_entropy = 0.802124
Epoch 212
Validation binary_cross_entropy = 0.814534
Epoch 213
Validation binary_cross_entropy = 0.841459
Epoch 214
Loss = 2.3263e-02, PNorm = 32.7507, GNorm = 2.2261, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.886668
Epoch 215
Validation binary_cross_entropy = 0.931071
Epoch 216
Validation binary_cross_entropy = 0.925877
Epoch 217
Validation binary_cross_entropy = 0.853135
Epoch 218
Validation binary_cross_entropy = 0.790386
Epoch 219
Loss = 2.5714e-02, PNorm = 32.7608, GNorm = 0.0952, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.763690
Epoch 220
Validation binary_cross_entropy = 0.752821
Epoch 221
Validation binary_cross_entropy = 0.750281
Epoch 222
Validation binary_cross_entropy = 0.751620
Epoch 223
Validation binary_cross_entropy = 0.791401
Epoch 224
Loss = 7.5663e-03, PNorm = 32.7705, GNorm = 0.3439, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.842972
Epoch 225
Validation binary_cross_entropy = 0.850875
Epoch 226
Validation binary_cross_entropy = 0.845344
Epoch 227
Validation binary_cross_entropy = 0.838333
Epoch 228
Validation binary_cross_entropy = 0.821380
Epoch 229
Loss = 3.0265e-02, PNorm = 32.7820, GNorm = 2.7510, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.829978
Epoch 230
Validation binary_cross_entropy = 0.846168
Epoch 231
Validation binary_cross_entropy = 0.856019
Epoch 232
Validation binary_cross_entropy = 0.850525
Epoch 233
Validation binary_cross_entropy = 0.826055
Epoch 234
Loss = 2.0013e-03, PNorm = 32.7941, GNorm = 0.2340, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.812665
Epoch 235
Validation binary_cross_entropy = 0.803115
Epoch 236
Validation binary_cross_entropy = 0.795029
Epoch 237
Validation binary_cross_entropy = 0.785366
Epoch 238
Validation binary_cross_entropy = 0.792853
Epoch 239
Loss = 3.5754e-03, PNorm = 32.8031, GNorm = 0.1373, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.807977
Epoch 240
Validation binary_cross_entropy = 0.829047
Epoch 241
Validation binary_cross_entropy = 0.832136
Epoch 242
Validation binary_cross_entropy = 0.822276
Epoch 243
Validation binary_cross_entropy = 0.815040
Epoch 244
Loss = 2.7084e-02, PNorm = 32.8103, GNorm = 0.3677, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.825265
Epoch 245
Validation binary_cross_entropy = 0.832283
Epoch 246
Validation binary_cross_entropy = 0.843974
Epoch 247
Validation binary_cross_entropy = 0.853315
Epoch 248
Validation binary_cross_entropy = 0.861192
Epoch 249
Loss = 1.3807e-02, PNorm = 32.8159, GNorm = 0.0361, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.850611
Epoch 250
Validation binary_cross_entropy = 0.850633
Epoch 251
Validation binary_cross_entropy = 0.842024
Epoch 252
Validation binary_cross_entropy = 0.834583
Epoch 253
Validation binary_cross_entropy = 0.842663
Epoch 254
Loss = 1.1472e-03, PNorm = 32.8221, GNorm = 0.0676, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.852591
Epoch 255
Validation binary_cross_entropy = 0.868063
Epoch 256
Validation binary_cross_entropy = 0.886864
Epoch 257
Validation binary_cross_entropy = 0.903634
Epoch 258
Validation binary_cross_entropy = 0.921402
Epoch 259
Loss = 1.5884e-02, PNorm = 32.8275, GNorm = 0.0725, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.922142
Epoch 260
Validation binary_cross_entropy = 0.917001
Epoch 261
Validation binary_cross_entropy = 0.915626
Epoch 262
Validation binary_cross_entropy = 0.918068
Epoch 263
Validation binary_cross_entropy = 0.914097
Epoch 264
Loss = 1.0393e-03, PNorm = 32.8353, GNorm = 0.0606, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.906617
Epoch 265
Validation binary_cross_entropy = 0.896558
Epoch 266
Validation binary_cross_entropy = 0.903690
Epoch 267
Validation binary_cross_entropy = 0.919418
Epoch 268
Validation binary_cross_entropy = 0.929379
Epoch 269
Loss = 3.1791e-02, PNorm = 32.8422, GNorm = 0.0622, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.956985
Epoch 270
Validation binary_cross_entropy = 0.955023
Epoch 271
Validation binary_cross_entropy = 0.935365
Epoch 272
Validation binary_cross_entropy = 0.899035
Epoch 273
Validation binary_cross_entropy = 0.864895
Epoch 274
Loss = 4.5787e-03, PNorm = 32.8491, GNorm = 0.6773, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.840130
Epoch 275
Validation binary_cross_entropy = 0.808735
Epoch 276
Validation binary_cross_entropy = 0.795030
Epoch 277
Validation binary_cross_entropy = 0.800539
Epoch 278
Validation binary_cross_entropy = 0.827960
Epoch 279
Loss = 4.6099e-03, PNorm = 32.8568, GNorm = 0.3679, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.863324
Epoch 280
Validation binary_cross_entropy = 0.901832
Epoch 281
Validation binary_cross_entropy = 0.935704
Epoch 282
Validation binary_cross_entropy = 0.961365
Epoch 283
Validation binary_cross_entropy = 0.967943
Epoch 284
Loss = 1.1673e-03, PNorm = 32.8632, GNorm = 0.0602, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.961774
Epoch 285
Validation binary_cross_entropy = 0.955724
Epoch 286
Validation binary_cross_entropy = 0.949493
Epoch 287
Validation binary_cross_entropy = 0.942632
Epoch 288
Validation binary_cross_entropy = 0.952279
Epoch 289
Loss = 2.5311e-03, PNorm = 32.8671, GNorm = 0.3268, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.958064
Epoch 290
Validation binary_cross_entropy = 0.953456
Epoch 291
Validation binary_cross_entropy = 0.928897
Epoch 292
Validation binary_cross_entropy = 0.908168
Epoch 293
Validation binary_cross_entropy = 0.917270
Epoch 294
Loss = 3.0354e-03, PNorm = 32.8720, GNorm = 0.0915, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.932033
Epoch 295
Validation binary_cross_entropy = 0.947709
Epoch 296
Validation binary_cross_entropy = 0.952442
Epoch 297
Validation binary_cross_entropy = 0.943591
Epoch 298
Validation binary_cross_entropy = 0.923433
Epoch 299
Loss = 8.4235e-03, PNorm = 32.8768, GNorm = 0.0157, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.897172
Model 0 best validation binary_cross_entropy = 0.283089 on epoch 1
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.277296
Ensemble test binary_cross_entropy = 0.277296
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1200, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 641,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.300938
Epoch 1
Validation binary_cross_entropy = 0.275312
Epoch 2
Validation binary_cross_entropy = 1.108929
Epoch 3
Validation binary_cross_entropy = 0.602908
Epoch 4
Loss = 6.3960e-01, PNorm = 31.6254, GNorm = 8.5110, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.291973
Epoch 5
Validation binary_cross_entropy = 0.410742
Epoch 6
Validation binary_cross_entropy = 0.715709
Epoch 7
Validation binary_cross_entropy = 0.444860
Epoch 8
Validation binary_cross_entropy = 0.374061
Epoch 9
Loss = 4.2593e-01, PNorm = 31.6829, GNorm = 3.5908, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.965951
Epoch 10
Validation binary_cross_entropy = 0.411925
Epoch 11
Validation binary_cross_entropy = 0.363991
Epoch 12
Validation binary_cross_entropy = 0.381461
Epoch 13
Validation binary_cross_entropy = 0.728559
Epoch 14
Loss = 3.0533e-01, PNorm = 31.7583, GNorm = 6.2382, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.641240
Epoch 15
Validation binary_cross_entropy = 0.451130
Epoch 16
Validation binary_cross_entropy = 0.393274
Epoch 17
Validation binary_cross_entropy = 0.396113
Epoch 18
Validation binary_cross_entropy = 0.463903
Epoch 19
Loss = 1.9565e-01, PNorm = 31.8210, GNorm = 2.0910, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.479425
Epoch 20
Validation binary_cross_entropy = 0.431079
Epoch 21
Validation binary_cross_entropy = 0.441180
Epoch 22
Validation binary_cross_entropy = 0.464327
Epoch 23
Validation binary_cross_entropy = 0.464462
Epoch 24
Loss = 1.8915e-01, PNorm = 31.8715, GNorm = 3.1025, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.436887
Epoch 25
Validation binary_cross_entropy = 0.415906
Epoch 26
Validation binary_cross_entropy = 0.414529
Epoch 27
Validation binary_cross_entropy = 0.431739
Epoch 28
Validation binary_cross_entropy = 0.434348
Epoch 29
Loss = 1.1994e-01, PNorm = 31.9123, GNorm = 2.6541, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.477564
Epoch 30
Validation binary_cross_entropy = 0.512562
Epoch 31
Validation binary_cross_entropy = 0.499565
Epoch 32
Validation binary_cross_entropy = 0.502416
Epoch 33
Validation binary_cross_entropy = 0.461145
Epoch 34
Loss = 2.0957e-01, PNorm = 31.9499, GNorm = 3.7224, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.464062
Epoch 35
Validation binary_cross_entropy = 0.488692
Epoch 36
Validation binary_cross_entropy = 0.444341
Epoch 37
Validation binary_cross_entropy = 0.528335
Epoch 38
Validation binary_cross_entropy = 0.589553
Epoch 39
Loss = 1.2401e-01, PNorm = 31.9879, GNorm = 1.6688, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.581520
Epoch 40
Validation binary_cross_entropy = 0.559361
Epoch 41
Validation binary_cross_entropy = 0.487835
Epoch 42
Validation binary_cross_entropy = 0.475504
Epoch 43
Validation binary_cross_entropy = 0.495323
Epoch 44
Loss = 3.2254e-01, PNorm = 32.0268, GNorm = 3.2895, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.504140
Epoch 45
Validation binary_cross_entropy = 0.553188
Epoch 46
Validation binary_cross_entropy = 0.510179
Epoch 47
Validation binary_cross_entropy = 0.590595
Epoch 48
Validation binary_cross_entropy = 0.607515
Epoch 49
Loss = 1.5452e-01, PNorm = 32.0697, GNorm = 3.0958, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.505478
Epoch 50
Validation binary_cross_entropy = 0.520440
Epoch 51
Validation binary_cross_entropy = 0.513245
Epoch 52
Validation binary_cross_entropy = 0.498826
Epoch 53
Validation binary_cross_entropy = 0.511836
Epoch 54
Loss = 1.3404e-01, PNorm = 32.1096, GNorm = 4.1502, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.496609
Epoch 55
Validation binary_cross_entropy = 0.509018
Epoch 56
Validation binary_cross_entropy = 0.534178
Epoch 57
Validation binary_cross_entropy = 0.532507
Epoch 58
Validation binary_cross_entropy = 0.519281
Epoch 59
Loss = 4.6659e-02, PNorm = 32.1442, GNorm = 1.7048, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.529971
Epoch 60
Validation binary_cross_entropy = 0.520741
Epoch 61
Validation binary_cross_entropy = 0.522812
Epoch 62
Validation binary_cross_entropy = 0.562589
Epoch 63
Validation binary_cross_entropy = 0.565145
Epoch 64
Loss = 7.1856e-02, PNorm = 32.1742, GNorm = 0.8920, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.525911
Epoch 65
Validation binary_cross_entropy = 0.523576
Epoch 66
Validation binary_cross_entropy = 0.532311
Epoch 67
Validation binary_cross_entropy = 0.544127
Epoch 68
Validation binary_cross_entropy = 0.568113
Epoch 69
Loss = 6.9657e-02, PNorm = 32.2029, GNorm = 2.1442, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.606023
Epoch 70
Validation binary_cross_entropy = 0.564324
Epoch 71
Validation binary_cross_entropy = 0.523273
Epoch 72
Validation binary_cross_entropy = 0.523678
Epoch 73
Validation binary_cross_entropy = 0.525090
Epoch 74
Loss = 1.0110e-01, PNorm = 32.2304, GNorm = 0.7686, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.550748
Epoch 75
Validation binary_cross_entropy = 0.612824
Epoch 76
Validation binary_cross_entropy = 0.611603
Epoch 77
Validation binary_cross_entropy = 0.542905
Epoch 78
Validation binary_cross_entropy = 0.530844
Epoch 79
Loss = 8.9255e-02, PNorm = 32.2568, GNorm = 2.8047, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.555899
Epoch 80
Validation binary_cross_entropy = 0.571120
Epoch 81
Validation binary_cross_entropy = 0.596281
Epoch 82
Validation binary_cross_entropy = 0.609891
Epoch 83
Validation binary_cross_entropy = 0.597550
Epoch 84
Loss = 3.8637e-02, PNorm = 32.2825, GNorm = 0.4844, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.580583
Epoch 85
Validation binary_cross_entropy = 0.557933
Epoch 86
Validation binary_cross_entropy = 0.539469
Epoch 87
Validation binary_cross_entropy = 0.536380
Epoch 88
Validation binary_cross_entropy = 0.533069
Epoch 89
Loss = 3.2068e-02, PNorm = 32.3037, GNorm = 1.4145, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.542702
Epoch 90
Validation binary_cross_entropy = 0.564142
Epoch 91
Validation binary_cross_entropy = 0.573225
Epoch 92
Validation binary_cross_entropy = 0.570493
Epoch 93
Validation binary_cross_entropy = 0.590719
Epoch 94
Loss = 5.1800e-02, PNorm = 32.3226, GNorm = 3.0212, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.598887
Epoch 95
Validation binary_cross_entropy = 0.570545
Epoch 96
Validation binary_cross_entropy = 0.560943
Epoch 97
Validation binary_cross_entropy = 0.557141
Epoch 98
Validation binary_cross_entropy = 0.551778
Epoch 99
Loss = 7.0530e-02, PNorm = 32.3477, GNorm = 2.8987, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.559968
Epoch 100
Validation binary_cross_entropy = 0.597026
Epoch 101
Validation binary_cross_entropy = 0.606756
Epoch 102
Validation binary_cross_entropy = 0.624249
Epoch 103
Validation binary_cross_entropy = 0.667345
Epoch 104
Loss = 2.6940e-02, PNorm = 32.3730, GNorm = 0.1431, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.713856
Epoch 105
Validation binary_cross_entropy = 0.702935
Epoch 106
Validation binary_cross_entropy = 0.672556
Epoch 107
Validation binary_cross_entropy = 0.642895
Epoch 108
Validation binary_cross_entropy = 0.615610
Epoch 109
Loss = 2.0400e-02, PNorm = 32.3957, GNorm = 1.3541, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.592443
Epoch 110
Validation binary_cross_entropy = 0.595584
Epoch 111
Validation binary_cross_entropy = 0.607930
Epoch 112
Validation binary_cross_entropy = 0.618799
Epoch 113
Validation binary_cross_entropy = 0.611418
Epoch 114
Loss = 6.1578e-02, PNorm = 32.4140, GNorm = 2.7754, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.598957
Epoch 115
Validation binary_cross_entropy = 0.595349
Epoch 116
Validation binary_cross_entropy = 0.603353
Epoch 117
Validation binary_cross_entropy = 0.620313
Epoch 118
Validation binary_cross_entropy = 0.637437
Epoch 119
Loss = 5.3111e-02, PNorm = 32.4315, GNorm = 3.9459, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.619643
Epoch 120
Validation binary_cross_entropy = 0.587146
Epoch 121
Validation binary_cross_entropy = 0.576896
Epoch 122
Validation binary_cross_entropy = 0.594118
Epoch 123
Validation binary_cross_entropy = 0.609204
Epoch 124
Loss = 3.1436e-02, PNorm = 32.4489, GNorm = 2.1169, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.631609
Epoch 125
Validation binary_cross_entropy = 0.667891
Epoch 126
Validation binary_cross_entropy = 0.687151
Epoch 127
Validation binary_cross_entropy = 0.686751
Epoch 128
Validation binary_cross_entropy = 0.682139
Epoch 129
Loss = 4.3965e-02, PNorm = 32.4650, GNorm = 0.5219, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.663815
Epoch 130
Validation binary_cross_entropy = 0.669802
Epoch 131
Validation binary_cross_entropy = 0.684908
Epoch 132
Validation binary_cross_entropy = 0.700281
Epoch 133
Validation binary_cross_entropy = 0.703292
Epoch 134
Loss = 1.6960e-02, PNorm = 32.4793, GNorm = 0.8350, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.694661
Epoch 135
Validation binary_cross_entropy = 0.685280
Epoch 136
Validation binary_cross_entropy = 0.685461
Epoch 137
Validation binary_cross_entropy = 0.693998
Epoch 138
Validation binary_cross_entropy = 0.730024
Epoch 139
Loss = 4.9555e-02, PNorm = 32.4914, GNorm = 3.1196, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.744996
Epoch 140
Validation binary_cross_entropy = 0.689691
Epoch 141
Validation binary_cross_entropy = 0.646770
Epoch 142
Validation binary_cross_entropy = 0.649115
Epoch 143
Validation binary_cross_entropy = 0.679639
Epoch 144
Loss = 2.8637e-02, PNorm = 32.5082, GNorm = 1.9975, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.709400
Epoch 145
Validation binary_cross_entropy = 0.752125
Epoch 146
Validation binary_cross_entropy = 0.757975
Epoch 147
Validation binary_cross_entropy = 0.738591
Epoch 148
Validation binary_cross_entropy = 0.713801
Epoch 149
Loss = 2.5032e-02, PNorm = 32.5256, GNorm = 1.8831, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.712998
Epoch 150
Validation binary_cross_entropy = 0.727469
Epoch 151
Validation binary_cross_entropy = 0.748533
Epoch 152
Validation binary_cross_entropy = 0.750968
Epoch 153
Validation binary_cross_entropy = 0.661198
Epoch 154
Loss = 1.0132e-02, PNorm = 32.5393, GNorm = 0.8756, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.622081
Epoch 155
Validation binary_cross_entropy = 0.628188
Epoch 156
Validation binary_cross_entropy = 0.638278
Epoch 157
Validation binary_cross_entropy = 0.678511
Epoch 158
Validation binary_cross_entropy = 0.697743
Epoch 159
Loss = 1.2486e-02, PNorm = 32.5583, GNorm = 0.9425, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.676621
Epoch 160
Validation binary_cross_entropy = 0.662005
Epoch 161
Validation binary_cross_entropy = 0.661030
Epoch 162
Validation binary_cross_entropy = 0.677172
Epoch 163
Validation binary_cross_entropy = 0.685075
Epoch 164
Loss = 3.8113e-03, PNorm = 32.5756, GNorm = 0.4186, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.689801
Epoch 165
Validation binary_cross_entropy = 0.685327
Epoch 166
Validation binary_cross_entropy = 0.678755
Epoch 167
Validation binary_cross_entropy = 0.686202
Epoch 168
Validation binary_cross_entropy = 0.715403
Epoch 169
Loss = 2.4811e-02, PNorm = 32.5873, GNorm = 1.6744, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.728992
Epoch 170
Validation binary_cross_entropy = 0.735962
Epoch 171
Validation binary_cross_entropy = 0.744731
Epoch 172
Validation binary_cross_entropy = 0.745727
Epoch 173
Validation binary_cross_entropy = 0.740426
Epoch 174
Loss = 1.0266e-02, PNorm = 32.5980, GNorm = 1.0435, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.752510
Epoch 175
Validation binary_cross_entropy = 0.769919
Epoch 176
Validation binary_cross_entropy = 0.792958
Epoch 177
Validation binary_cross_entropy = 0.840701
Epoch 178
Validation binary_cross_entropy = 0.865094
Epoch 179
Loss = 1.8799e-02, PNorm = 32.6084, GNorm = 0.1433, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.846932
Epoch 180
Validation binary_cross_entropy = 0.902080
Epoch 181
Validation binary_cross_entropy = 0.917576
Epoch 182
Validation binary_cross_entropy = 0.881659
Epoch 183
Validation binary_cross_entropy = 0.838651
Epoch 184
Loss = 5.4292e-02, PNorm = 32.6254, GNorm = 3.6623, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.835473
Epoch 185
Validation binary_cross_entropy = 0.866815
Epoch 186
Validation binary_cross_entropy = 0.850256
Epoch 187
Validation binary_cross_entropy = 0.796807
Epoch 188
Validation binary_cross_entropy = 0.741675
Epoch 189
Loss = 4.0429e-03, PNorm = 32.6465, GNorm = 0.1495, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.712164
Epoch 190
Validation binary_cross_entropy = 0.695302
Epoch 191
Validation binary_cross_entropy = 0.698279
Epoch 192
Validation binary_cross_entropy = 0.704072
Epoch 193
Validation binary_cross_entropy = 0.761736
Epoch 194
Loss = 5.6384e-03, PNorm = 32.6607, GNorm = 0.3501, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.828101
Epoch 195
Validation binary_cross_entropy = 0.840743
Epoch 196
Validation binary_cross_entropy = 0.804959
Epoch 197
Validation binary_cross_entropy = 0.795234
Epoch 198
Validation binary_cross_entropy = 0.828431
Epoch 199
Loss = 1.8472e-03, PNorm = 32.6731, GNorm = 0.1134, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.858967
Epoch 200
Validation binary_cross_entropy = 0.873537
Epoch 201
Validation binary_cross_entropy = 0.878541
Epoch 202
Validation binary_cross_entropy = 0.874040
Epoch 203
Validation binary_cross_entropy = 0.854916
Epoch 204
Loss = 1.5480e-02, PNorm = 32.6843, GNorm = 0.7563, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.850882
Epoch 205
Validation binary_cross_entropy = 0.856716
Epoch 206
Validation binary_cross_entropy = 0.932928
Epoch 207
Validation binary_cross_entropy = 0.968119
Epoch 208
Validation binary_cross_entropy = 1.036637
Epoch 209
Loss = 1.5864e-02, PNorm = 32.7017, GNorm = 1.5399, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.032900
Epoch 210
Validation binary_cross_entropy = 0.965531
Epoch 211
Validation binary_cross_entropy = 0.950032
Epoch 212
Validation binary_cross_entropy = 0.919618
Epoch 213
Validation binary_cross_entropy = 0.822728
Epoch 214
Loss = 2.6638e-02, PNorm = 32.7248, GNorm = 1.5306, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.820629
Epoch 215
Validation binary_cross_entropy = 0.919736
Epoch 216
Validation binary_cross_entropy = 1.014433
Epoch 217
Validation binary_cross_entropy = 0.956558
Epoch 218
Validation binary_cross_entropy = 0.888312
Epoch 219
Loss = 9.5848e-03, PNorm = 32.7551, GNorm = 0.0558, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.826498
Epoch 220
Validation binary_cross_entropy = 0.826889
Epoch 221
Validation binary_cross_entropy = 0.851895
Epoch 222
Validation binary_cross_entropy = 0.879339
Epoch 223
Validation binary_cross_entropy = 0.879665
Epoch 224
Loss = 5.4613e-02, PNorm = 32.7810, GNorm = 0.5426, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.861073
Epoch 225
Validation binary_cross_entropy = 0.869312
Epoch 226
Validation binary_cross_entropy = 0.855637
Epoch 227
Validation binary_cross_entropy = 0.826490
Epoch 228
Validation binary_cross_entropy = 0.810055
Epoch 229
Loss = 3.4548e-03, PNorm = 32.8030, GNorm = 0.3331, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.814500
Epoch 230
Validation binary_cross_entropy = 0.819184
Epoch 231
Validation binary_cross_entropy = 0.869740
Epoch 232
Validation binary_cross_entropy = 0.943718
Epoch 233
Validation binary_cross_entropy = 1.000904
Epoch 234
Loss = 5.1207e-02, PNorm = 32.8225, GNorm = 2.3359, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.960980
Epoch 235
Validation binary_cross_entropy = 0.894735
Epoch 236
Validation binary_cross_entropy = 0.863692
Epoch 237
Validation binary_cross_entropy = 0.879995
Epoch 238
Validation binary_cross_entropy = 0.887669
Epoch 239
Loss = 5.7864e-02, PNorm = 32.8477, GNorm = 3.0774, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.880020
Epoch 240
Validation binary_cross_entropy = 0.954038
Epoch 241
Validation binary_cross_entropy = 1.056134
Epoch 242
Validation binary_cross_entropy = 1.074575
Epoch 243
Validation binary_cross_entropy = 0.925064
Epoch 244
Loss = 5.6155e-03, PNorm = 32.8867, GNorm = 0.1807, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.000586
Epoch 245
Validation binary_cross_entropy = 1.128229
Epoch 246
Validation binary_cross_entropy = 1.078030
Epoch 247
Validation binary_cross_entropy = 0.890711
Epoch 248
Validation binary_cross_entropy = 0.895669
Epoch 249
Loss = 2.0959e-02, PNorm = 32.9373, GNorm = 1.1863, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.916933
Epoch 250
Validation binary_cross_entropy = 0.881266
Epoch 251
Validation binary_cross_entropy = 0.824360
Epoch 252
Validation binary_cross_entropy = 0.828686
Epoch 253
Validation binary_cross_entropy = 0.843574
Epoch 254
Loss = 2.7915e-02, PNorm = 32.9773, GNorm = 1.9185, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.829111
Epoch 255
Validation binary_cross_entropy = 0.833581
Epoch 256
Validation binary_cross_entropy = 0.847176
Epoch 257
Validation binary_cross_entropy = 0.880723
Epoch 258
Validation binary_cross_entropy = 0.893405
Epoch 259
Loss = 1.3690e-02, PNorm = 33.0092, GNorm = 0.1873, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.874289
Epoch 260
Validation binary_cross_entropy = 0.846380
Epoch 261
Validation binary_cross_entropy = 0.830540
Epoch 262
Validation binary_cross_entropy = 0.836887
Epoch 263
Validation binary_cross_entropy = 0.855061
Epoch 264
Loss = 3.2677e-02, PNorm = 33.0310, GNorm = 1.1280, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.867322
Epoch 265
Validation binary_cross_entropy = 0.857639
Epoch 266
Validation binary_cross_entropy = 0.868331
Epoch 267
Validation binary_cross_entropy = 0.904889
Epoch 268
Validation binary_cross_entropy = 0.928245
Epoch 269
Loss = 2.8325e-02, PNorm = 33.0450, GNorm = 0.2695, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.964333
Epoch 270
Validation binary_cross_entropy = 0.979165
Epoch 271
Validation binary_cross_entropy = 0.978025
Epoch 272
Validation binary_cross_entropy = 0.967658
Epoch 273
Validation binary_cross_entropy = 0.959009
Epoch 274
Loss = 4.3725e-03, PNorm = 33.0564, GNorm = 0.0342, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.956979
Epoch 275
Validation binary_cross_entropy = 0.953195
Epoch 276
Validation binary_cross_entropy = 0.945997
Epoch 277
Validation binary_cross_entropy = 0.940642
Epoch 278
Validation binary_cross_entropy = 0.938640
Epoch 279
Loss = 2.5949e-03, PNorm = 33.0646, GNorm = 0.1495, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.932131
Epoch 280
Validation binary_cross_entropy = 0.928047
Epoch 281
Validation binary_cross_entropy = 0.922383
Epoch 282
Validation binary_cross_entropy = 0.925945
Epoch 283
Validation binary_cross_entropy = 0.946172
Epoch 284
Loss = 5.0281e-03, PNorm = 33.0707, GNorm = 0.0528, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.967064
Epoch 285
Validation binary_cross_entropy = 0.948041
Epoch 286
Validation binary_cross_entropy = 0.930618
Epoch 287
Validation binary_cross_entropy = 0.907697
Epoch 288
Validation binary_cross_entropy = 0.874164
Epoch 289
Loss = 2.1514e-03, PNorm = 33.0802, GNorm = 0.0490, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.851191
Epoch 290
Validation binary_cross_entropy = 0.837115
Epoch 291
Validation binary_cross_entropy = 0.828885
Epoch 292
Validation binary_cross_entropy = 0.838794
Epoch 293
Validation binary_cross_entropy = 0.847643
Epoch 294
Loss = 8.0516e-03, PNorm = 33.0874, GNorm = 0.2491, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.854183
Epoch 295
Validation binary_cross_entropy = 0.856672
Epoch 296
Validation binary_cross_entropy = 0.857376
Epoch 297
Validation binary_cross_entropy = 0.860283
Epoch 298
Validation binary_cross_entropy = 0.870778
Epoch 299
Loss = 3.4230e-03, PNorm = 33.0934, GNorm = 0.3301, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.881262
Model 0 best validation binary_cross_entropy = 0.275312 on epoch 1
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.288185
Ensemble test binary_cross_entropy = 0.288185
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.214004
	Seed 1 ==> test binary_cross_entropy = 0.236231
	Seed 2 ==> test binary_cross_entropy = 0.220059
	Seed 3 ==> test binary_cross_entropy = 0.243466
	Seed 4 ==> test binary_cross_entropy = 0.243307
	Seed 5 ==> test binary_cross_entropy = 0.244644
	Seed 6 ==> test binary_cross_entropy = 0.279091
	Seed 7 ==> test binary_cross_entropy = 0.320771
	Seed 8 ==> test binary_cross_entropy = 0.277296
	Seed 9 ==> test binary_cross_entropy = 0.288185
Overall test binary_cross_entropy = 0.256706 +/- 0.031790
Elapsed time = 0:03:13
